Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41772

Early exhaustion of client-generator for set-delete workloads

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • master
    • master
    • Jepsen
    • None

    Description

      The options map in cbjcli provides a default value of 30 for :doc-count, which takes precedence over the :doc-count default of 10000 defined in the set-delete workloads. By default we therefore only perform 30 add operations and 15 delete operations for set-delete workloads. These all occur simultaneously at the start of the workload (since concurrency > 45), causing immediate exhaustion of the client-generator. This particularly affects the set-delete-kill workload, where we then perform those 45 ops before the nemesis ever gets a chance to do anything, breaking the entire puropse of the test.

      Even if the :doc-count would correctly default to 10000, we can still easily exhaust the client generator before the nemesis has completed the requested number of cycles. The client generator should therefore be modified to continuously yield operations until the nemesis has completed, in line with the behaviour of the other workloads.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Need to check that this MB is resolved before releasing Jepsen

          richard.demellow Richard deMellow added a comment - Need to check that this MB is resolved before releasing Jepsen
          asad.zaidi Asad Zaidi (Inactive) added a comment - - edited

          Looking at the code, the test has been amended such that its behaviour consistent with all of the other tests.

          The test is no longer configurable by :doc-count.

          The client generator will generate add operations with keys in the half-open interval [0 to 10000) followed by delete operations with keys in [0 to 5000) advancing the interval forward by 10000 once the previous 15000 add and delete operations have been generated.

          The client generator will generate operations until the specified number of nemesis cycles have completed.

          This seems to be true both for the 'set-delete' and 'set-delete-kill' workloads.

          There's also a 'set-kill' workload which is unaffected by these issues as it only generates add operations infinitely.

           

          Testing with 'set-delete' workload, the number items added seems to scale linearly with the number of cycles.

          I will go ahead and resolve this issue as this has already been fixed.

           

          asad.zaidi Asad Zaidi (Inactive) added a comment - - edited Looking at the code, the test has been amended such that its behaviour consistent with all of the other tests. The test is no longer configurable by :doc-count. The client generator will generate add operations with keys in the half-open interval [0 to 10000) followed by delete operations with keys in [0 to 5000) advancing the interval forward by 10000 once the previous 15000 add and delete operations have been generated. The client generator will generate operations until the specified number of nemesis cycles have completed. This seems to be true both for the 'set-delete' and 'set-delete-kill' workloads. There's also a 'set-kill' workload which is unaffected by these issues as it only generates add operations infinitely.   Testing with 'set-delete' workload, the number items added seems to scale linearly with the number of cycles. I will go ahead and resolve this issue as this has already been fixed.  

          Closing tickets

          richard.demellow Richard deMellow added a comment - Closing tickets

          People

            asad.zaidi Asad Zaidi (Inactive)
            sven.signer Sven Signer (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty