Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60936

Reduce contention from CheckpointManager memory usage CoreLocals

    XMLWordPrintable

Details

    • 0
    • Yes
    • March-June 24

    Description

      During every write operation, we will call KVBucket::getCheckpointMemoryState, which will check the CM memory usage via getCheckpointManagerEstimatedMemUsage and the pending destruction memory usage via getCheckpointPendingDestructionMemoryUsage.

      Both of these getters aggregate some CoreLocal counters on every call. This is sub-optimal on the getCheckpointMemoryState path, because any CoreLocal counts that have been invalidated in a core's cache will require cross-cache communication to read from the caller of getCheckpointMemoryState. Theoretically, this is worse than a simple std::atomic on the read path.

      Since all KV writes update a CoreLocal counter, and every write reads all CoreLocals, there can be a lot of cross-cache communication.

      For the bucket quota check, we keep an approximate estimate of the total bucket memory usage in a CoreLocal, and we use that estimate on the hot path, instead of summing up all of the corelocals. We should consider doing the same here, by refactoring the JEArenaCoreLocalTracker logic into a template and re-using for the CM.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavlos.georgiou Pavlos Georgiou
              vesko.karaganev Vesko Karaganev
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty