vikass-MacBook-Pro:~ vikaschaudhary$ python jenkins_log_parser.py http://qa.sc.couchbase.com/job/test_suite_executor/176478
|
1 test_create_mutation_for_dcp_stream_boundary_from_beginning
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1884}}
|
|
|
2 test_delete_mutation_for_dcp_stream_boundary_from_beginning
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1803}}
|
|
|
3 test_update_mutation_for_dcp_stream_boundary_from_now
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 2016}}
|
|
|
4 test_expiry_mutation_for_dcp_stream_boundary_from_beginning
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1885}}
|
|
|
7 test_doc_timer_events_from_handler_code_with_bucket_ops
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1802}}
|
|
|
9 test_cron_timer_events_from_handler_code_with_bucket_ops
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1802}}
|
|
|
10 test_delete_bucket_operation_from_handler_code
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1886}}
|
|
|
11 test_timers_without_context
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1773}}
|
|
|
12 test_cancel_timers_with_timers_being_overwritten
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1227}}
|
|
|
13 test_source_bucket_mutations_with_timers
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 2016 Expected : 4032 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1314}}
|
|
|
14 test_source_bucket_mutations
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 2016 Expected : 4032 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1886}}
|
|
|
15 test_source_doc_mutations
|
Exception: Eventing has not processed all the mutation in expected time, docs:0 expected doc: 2016
|
|
|
16 test_source_doc_mutations_with_timers
|
Exception: Eventing has not processed all the mutation in expected time, docs:0 expected doc: 2016
|
|
|
17 test_pause_resume_execution
|
Exception: Eventing took lot of time for handler Function_991174942_test_pause_resume_execution to paused
|
|
|
18 test_source_bucket_mutation_for_dcp_stream_boundary_from_now
|
Exception: Eventing has not processed all the dcp_mutation. Current : 2016 Expected : 4032
|
|
|
19 test_compress_handler
|
Exception: Bucket operations from handler code took lot of time to complete or didn't go through. Current : 0 Expected : 2016 dcp_backlog : 0 TIMERS_IN_PAST : 0 lcb_exceptions : {'[fd63:6f75:6368:20d4:b164:126b:5258:cac6]': {u'27': 1887}}
|
Bucket op failing only on IPv6 cluster.
Eventing log shows:
2019-11-25T16:38:44.008+05:30 [Info] eventing-consumer [worker_test_2:/tmp/[::1]:9000_2_3746722029.sock:19776] [lcb,connection L:156 I:4186126171] <[::1:9000]:8091> (SOCK=ef3d9144d42eda7f) Failed to establish connection: LCB_ENETUNREACH (0x31), os errno=101
2019-11-25T16:38:44.008+05:30 [Info] eventing-consumer [worker_test_2:/tmp/[::1]:9000_2_3746722029.sock:19776] [lcb,htconfig L:301 I:4186126171] Connection to REST API failed with LCB_ENETUNREACH (0x31) (os errno = 101)
2019-11-25T16:38:44.008+05:30 [Info] eventing-consumer [worker_test_2:/tmp/[::1]:9000_2_3746722029.sock:19776] [lcb,confmon L:179 I:4186126171] Provider 'HTTP' failed
2019-11-25T16:38:44.008+05:30 [Info] eventing-consumer [worker_test_2:/tmp/[::1]:9000_2_3746722029.sock:19776] [lcb,bootstrap L:170 I:4186126171] Failed to bootstrap client=0x7fa4f00cd680. Error=LCB_ENETUNREACH (0x31), Message=No more bootstrap providers remain
Eventing application log shows:
2019-11-25T16:42:19.924+05:30 [INFO] {"message":{"code":8,"desc":"Client not bootstrapped. Ensure bootstrap/connect was attempted and was successful","name":"LCB_CLIENT_ENOCONF"}
I observed that this happens after the call to * lcb_wait* fails. We do retry 5 times from the eventing side but still we are not able to get the connection.