Details
Major Description
This happened during build 4.5.0-1733. There were no forestdb changes from the last time unit tests ran (4.5.0-1730). So possibly some kind of env issue or some condition the the test doesn't handle.
Email conversation with Sundar attached below:
Yes, it is possible, because the host’s load is pretty high during the build (these are docker containers sitting on a single host — i.e, 5 different linux builds are fired off on 5 different containers sitting on the same host).
|
|
|
Do we need such a heavy test for “unit tests”? If they are needed, can we try make it such that it can recover from period high load on the host? The fdb_functional_test for these two platforms start about 30 seconds apart. They both time out at around the same time. Is it possible both the platforms run the same test at the same time and are throttling each other?
|
|
|
Thanks,
|
-Hari
|
|
|
|
|
From: Sundar Sridharan <Sundar@couchbase.com>
|
Date: Tuesday, March 8, 2016 at 12:35 PM
|
To: Hari Kodungallur <hari.kodungallur@couchbase.com>, Chiyoung Seo <chiyoung@couchbase.com>
|
Cc: Keshav Murthy <keshav@couchbase.com>, build-team <build-team@couchbase.com>, Raju Suravarjjala <raju@couchbase.com>, Jung-Sang Ahn <Jung-Sang@couchbase.com>
|
Subject: Re: fdb-functional-test timeout
|
|
|
Hi Hari,
|
|
|
Do we know if the disk on the machine is maxed out in its write cycles? It appears in both cases of failure the last test to have run is the large batch write test which creates a huge file on the disk.
|
|
|
Thanks and with warm regards,
|
Sundar
|
|
|
From: Hari Kodungallur <hari.kodungallur@couchbase.com>
|
Date: Tuesday, March 8, 2016 at 12:29 PM
|
To: Chiyoung Seo <chiyoung@couchbase.com>, Sundar Sridharan <sundar@couchbase.com>
|
Cc: Keshav Murthy <keshav@couchbase.com>, build-team <build-team@couchbase.com>, Raju Suravarjjala <raju@couchbase.com>
|
Subject: fdb-functional-test timeout
|
|
|
Hi Chiyoung / Sundar,
|
|
|
Can you please check these builds:
|
http://server.jenkins.couchbase.com/job/watson-unix/12901/console
|
http://server.jenkins.couchbase.com/job/watson-unix/12903/console
|
|
|
It looks like the fdb_functional_test timed out and aborted the build as it took way too much time.
|
Do you know why this could happen?
|
|
|
Note there were no forestdb changes since the last unit test run which was 4.5.0-1730 and this issues was on 4.5.0-1733. So, it is either some env issues or some “pre-existing condition” that got triggered somehow.
|
|
|
Thanks,
|
-Hari
|