Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19639

engine_testapp: Allow >1 attempt for a test to pass

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 3.1.5, 4.1.1, 4.5.0
    • 3.1.6, 4.1.2, 4.5.0
    • memcached
    • None

    Description

      We (unfortunately) have some tests driven by engine_testapp which intermittently fail - for example ep_testsuite. While much has been done to improve these test issues (particulary on newer branches), the fact remains that some tests are flaky and need retrying to pass.

      For commit-validation jobs, this means that the whole job will fail, and has to be manually retried. Not only is the repeat job time-consuming (CV job runtimes vary, but say ~10mins on average), but it also requires human intervention to trigger the retry.

      To attempt to improve this, we should add the ability for engine_testapp to attempt a
      test multiple times before declaring it a failure. For branches under maintenance (3.1.x, sherlock) which we are unlikely to spend much effort in fixing intermittent failures, this should significantly speed up our ability to get changes through CV.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            drigby Dave Rigby added a comment -

            Patch submitted to memcached/3.0: http://review.couchbase.org/#/c/64047/

            drigby Dave Rigby added a comment - Patch submitted to memcached/3.0: http://review.couchbase.org/#/c/64047/

            Dave Rigby Is it ok to close this one?

            ericcooper Eric Cooper (Inactive) added a comment - Dave Rigby Is it ok to close this one?

            People

              drigby Dave Rigby
              drigby Dave Rigby
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty