Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37643

Max TTL is incorrectly applied if it exceeds 30 days

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 6.0.0, 5.5.0, 5.5.1, 5.5.2, 5.5.3, 6.0.1, 6.0.2, 5.5.4, 6.0.3, 5.5.5, 5.5.6
    • Fix Version/s: 6.0.4
    • Component/s: couchbase-bucket
    • Labels:
      None
    • Triage:
      Untriaged
    • Is this a Regression?:
      No

      Description

      The max TTL for an item is applied incorrectly if the max TTL exceeds 30 days.
      Instead of the max TTL being applied as an offset from the current time, it is instead applied as an offset from when memcached started.

      Steps To Reproduce
      1. Start up Couchbase Server on a single node (this is time t start).
      2. Create a bucket with max TTL of 2592001 (30 days + 1 second).
      3. Create a document in the bucket with no expiry, e.g. from the UI (this is time t doc).
      4. Check the expiry (via couch_dbdump, the UI etc).

      Expected Result
      Document has expiry of t doc + 2592001.

      Actual Result
      Document has expiry of t start + 2592001, every single document created in this bucket (with no explicit expiry set) has the exact same expiry, no matter what time they were created.

      Analysis

      This happens because there is an issue in the method getExpiryParameters() (http://src.couchbase.org/source/xref/6.0.2/kv_engine/engines/ep/src/ep_engine.cc#2224):

       std::pair<cb::ExpiryLimit, rel_time_t>
      EventuallyPersistentEngine::getExpiryParameters(rel_time_t exptime) const {
          cb::ExpiryLimit expiryLimit;
          auto limit = kvBucket->getMaxTtl();
          if (limit.count()) {
              expiryLimit = limit;
              // If max_ttl is more than 30 days we need to convert it to absolute so
              // it makes sense as an expiry time.
              if (exptime == 0) {
                  if (limit.count() > (60 * 60 * 24 * 30)) {
                      exptime = ep_abs_time(limit.count());
                  } else {
                      exptime = limit.count();
                  }
              }
          }
       
          return {expiryLimit, exptime};
      }
      

      Here, if the max ttl is greater than 30 days we have to convert it to an absolute time (based on Memcached's epoch), to do this we use ep_abs_time().

      Below is the definition of mc_time_convert_to_abs_time() (http://src.couchbase.org/source/xref/6.0.2/kv_engine/daemon/mc_time.cc#181-183) which ep_abs_time() is mapped to:

      time_t mc_time_convert_to_abs_time(const rel_time_t rel_time) {
          return memcached_epoch + rel_time;
      }
      

      This returns the memcached epoch (which is the time that memcached started) plus the relative time.

      This means in the case above, we are setting the expiry to memcached start time plus the max ttl, instead of the current time.

      The correct code for this case would instead be (to be consistent with other uses of ep_abs_time() within the codebase):

      exptime = ep_abs_time(ep_current_time() + limit.count());
      

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            anitha.kuberan Anitha Kuberan added a comment -

            Verified on 6.0.4-3082

            Steps:

            1. create a bucket with TTL as 2592001 
            2. create a doc with no expiry and check expiry in couch_dbdump 
            3. check if the doc expiry is doc creation time + 2592001

            /opt/couchbase/bin/couch_dbdump 140.couch.1

            Dumping "140.couch.1":

            Doc seq: 1

                 id: doc2

                 rev: 1

                 content_meta: 128

                 size (on disk): 80

                 cas: 1580801741583220736, expiry: 1583393741, flags: 33554438, datatype: 0x01 (json)

                 size: 72

                 data: (snappy)

            { "click": "to edit", "with JSON": "there are no reserved field names" }

             

            Total docs: 1

             

             

            Show
            anitha.kuberan Anitha Kuberan added a comment - Verified on 6.0.4-3082 Steps: create a bucket with TTL as 2592001  create a doc with no expiry and check expiry in couch_dbdump  check if the doc expiry is doc creation time + 2592001 /opt/couchbase/bin/couch_dbdump 140.couch.1 Dumping "140.couch.1": Doc seq: 1      id: doc2      rev: 1      content_meta: 128      size (on disk): 80      cas: 1580801741583220736, expiry: 1583393741, flags: 33554438, datatype: 0x01 (json)      size: 72      data: (snappy) { "click": "to edit", "with JSON": "there are no reserved field names" }   Total docs: 1    
            Hide
            owend Daniel Owen added a comment - - edited

            opening to update Affects version and fix versions
            No update required as 6.5.0 is not affected.

            Show
            owend Daniel Owen added a comment - - edited opening to update Affects version and fix versions No update required as 6.5.0 is not affected.
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7519 contains kv_engine commit a8e72a7 with commit message:
            Merge "MB-37643: Merge branch 'couchbase/alice' into mad-hatter" into mad-hatter

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7519 contains kv_engine commit a8e72a7 with commit message: Merge " MB-37643 : Merge branch 'couchbase/alice' into mad-hatter" into mad-hatter
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7519 contains kv_engine commit ffe3eef with commit message:
            MB-37643: Merge branch 'couchbase/alice' into mad-hatter

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7519 contains kv_engine commit ffe3eef with commit message: MB-37643 : Merge branch 'couchbase/alice' into mad-hatter
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7519 contains kv_engine commit c4ede32 with commit message:
            MB-37643: Set expiry relative to uptime

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7519 contains kv_engine commit c4ede32 with commit message: MB-37643 : Set expiry relative to uptime

              People

              Assignee:
              matt.carabine Matt Carabine
              Reporter:
              matt.carabine Matt Carabine
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty