Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49727

Expiry ignores CheckpointManager quota

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.1.0
    • 7.1.0
    • couchbase-bucket
    • None
    • Untriaged
    • 1
    • No
    • KV 2021-Dec, KV 2022-Jan

    Description

      Items may be expired by compaction, or for stored values which are in the hashtable, by a PagingVisitor (created by ExpiredItemPager and ItemPager).

      (They may also be expired on access, but this is not discussed here).

      The callback used during compaction to expire items checks memory usage is below a configureable quota.

      class ExpiredItemsCallback : public Callback<Item&, time_t&> {
      ...
          void callback(Item& it, time_t& startTime) override {
              if (epstore.compactionCanExpireItems()) {
                  epstore.deleteExpiredItem(it, startTime, ExpireBy::Compactor);
              }
          }
      

      This prevents compaction driving memory usage arbitrarily high; the default threshold is 85%, the same as the default high watermark.

      Expiry via PagingVisitor does not make any checks before expiring items.

      In Neo, the CheckpointManager quota has been introduced. This is used to limit the maximum memory CheckpointManagers for all vbuckets can consume.

      As this is not checked in either expiry path, checkpoint memory usage may be driven beyond the CM quota (if it cannot be reduced by checkpoint removai/expelling/cursor dropping fast enough).

      Expiry through either compaction or visitor should consider the state of the CM, as a front end op would.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              Balakumaran.Gopal Balakumaran Gopal
              james.harrison James Harrison (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty