Details
-
Task
-
Resolution: Unresolved
-
Major
-
None
Description
Caught by jenkins_failures.py as the cause of some recent failures.
+Error signature+
|
|
|
Timeout of a CTest test N/M Test X: ep-engine_ep_unit_tests.CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly ...}
|
|
|
+Details+
|
11 instances of this failure (16.2% of sampled failures): |
* Time: 2024-01-11 09:45:47, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29174/, patch: https://review.couchbase.org/c/kv_engine/+/202747 |
* Time: 2024-01-02 11:29:28, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29103/, patch: https://review.couchbase.org/c/kv_engine/+/203092 |
* Time: 2023-12-18 11:14:04, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29076/, patch: https://review.couchbase.org/c/kv_engine/+/202687 |
* Time: 2023-12-15 16:11:00, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29054/, patch: https://review.couchbase.org/c/kv_engine/+/202687 |
* Time: 2023-12-15 09:20:43, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29038/, patch: https://review.couchbase.org/c/kv_engine/+/202668 |
* Time: 2023-12-14 09:50:38, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29016/, patch: https://review.couchbase.org/c/kv_engine/+/202554 |
* Time: 2024-01-10 11:54:29, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.ASan-UBSan/job/master/37073/, patch: https://review.couchbase.org/c/kv_engine/+/203384 |
* Time: 2024-01-10 11:21:30, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.ASan-UBSan/job/master/37072/, patch: https://review.couchbase.org/c/kv_engine/+/203384 |
* Time: 2024-01-02 05:38:52, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.ASan-UBSan/job/master/37010/, patch: https://review.couchbase.org/c/kv_engine/+/203034 |
* Time: 2023-12-14 13:15:41, Jenkins job: https://cv.jenkins.couchbase.com/job/kv_engine.threadsanitizer/job/master/30084/, patch: https://review.couchbase.org/c/kv_engine/+/202617 |
<cut> - only showing details of first 10/11 instances. |
|
We have a timeout for tests which is 5 minutes. The timeout is specified per test suite, and applied per test suite.
In this case, it appears that the entire run of the CollectionsDcpParameterizedTest test suite is taking longer than 5 minutes, and timing out.
From the logs in https://cv.jenkins.couchbase.com/job/kv_engine.linux/job/master/29038/consoleFull, we can several parameterised runs of the same test taking a total of 110s:
rg '(\d{5,} ms)' /tmp/output.txt -N |
10:14:48 [ OK ] CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly.create_drop_create_same_id_update/magma_persistent_value_only_ (11217 ms) |
10:14:48 [ OK ] CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly.create_drop_create_same_id_update/magma_persistent_value_only_per_doc_comp (26504 ms) |
10:14:48 [ OK ] CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly.create_drop_create_same_id_end_dropped_update/magma_persistent_value_only_per_doc_comp (30659 ms) |
10:14:48 [ OK ] CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly.create_drop_create_same_id_update_delete/nexus_persistent_full_eviction__couchstore_magma (31052 ms) |
10:14:48 [ OK ] CollectionsDcpEphemeralOrPersistent/CollectionsDcpPersistentOnly.create_drop_create_same_id_end_dropped_update_delete/nexus_persistent_full_eviction_per_doc_comp_couchstore_magma (13661 ms) |
We need to investigate why a create_drop_create_same_id_end_dropped_update run on Magma can take 30s.