Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19738

Incredibly slow unit test runs on Docker build slave

    XMLWordPrintable

Details

    • Untriaged
    • Ubuntu 64-bit
    • Unknown

    Description

      I had a commit-validation job timeout today - http://cv.jenkins.couchbase.com/job/ep-engine-threadsanitizer-master/760/consoleFull#2002939979c5751633-7aa5-473a-91e8-426e0afa5a7b Looking at the build log, I saw that the test which failed timed out after waiting for a file to exist (after 5 seconds).

      This is a generous timeout - the test takes 53ms on my local Linux server (24core Xeon, spinning disks):

      [ RUN      ] FullAndValueEviction/EPStoreEvictionTest.GetKeyStatsEjected/value_only
      [       OK ] FullAndValueEviction/EPStoreEvictionTest.GetKeyStatsEjected/value_only (53 ms)
      

      Whereas on the build slave:

      [ RUN      ] FullAndValueEviction/EPStoreEvictionTest.GetKeyStatsEjected/value_only
      /home/couchbase/jenkins/workspace/ep-engine-threadsanitizer-master/ep-engine/tests/module_tests/evp_store_test.cc:186: Failure
      Value of: result
      Actual: -1
      Expected: 1
      Failed to flush the one item we have stored.
      [  FAILED  ] FullAndValueEviction/EPStoreEvictionTest.GetKeyStatsEjected/value_only, where GetParam() = "value_only" (6375 ms)
      

      (Note this test is running under threadSanitizer, but that's true in both environments.)

      I don't have an issue with tests running a bit slower on the commit-validation machines, but over two orders of magnitude seems excessive.

      Could someone from build / IT take a look at this?

      Attachments

        Activity

          People

            hari.kodungallur hari kodungallur (Inactive)
            drigby Dave Rigby (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty