Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49978

Expiry pager uses stale time

    XMLWordPrintable

Details

    • 1
    • KV 2023-4

    Description

      PagingVisitor stores the current time at creation, and while visiting items compares them against that time.

      kv_engine/engines/ep/src/paging_visitor.cc, PagingVisitor::visit

      72
      bool isExpired = (currentBucket->getState() == vbucket_state_active) &&
      73
                           v.isExpired(startTime) && !v.isDeleted();
      

      kv_engine/engines/ep/src/paging_visitor.cc, PagingVisitor::visit

      36
      PagingVisitor::PagingVisitor(...) :
      37
      ...
      38
            startTime(ep_real_time()),
      

      Anecdotally, for large (~200GB) buckets with an expiry-heavy workload, the paging visitor has been seen to run for 1h30m.

      In an ephemeral bucket with low purge intervals, this may mean logically expired items might not be deleted until the current expiry pager run ends and the next begins, delaying memory recovery.

      Checking the current time for every item may be more expensive; a good compromise may be to update the cached time before visiting each vbucket.

      Attachments

        For Gerrit Dashboard: MB-49978
        # Subject Branch Project Status CR V

        Activity

          People

            owend Daniel Owen
            james.harrison James Harrison (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There is 1 open Gerrit change

                PagerDuty