Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50195

Magma memory defragmenter

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • Morpheus
    • Morpheus
    • storage-engine
    • 1

    Description

      In MB-49811, it was observed that BlockCache gets fragmentation after the workload stops.

      In that workload, deallocation of other objects that shared bins with BlockCache objects caused it to get fragmented. BlockCache objects are long-lived and their reallocation will only happen in an active workload. In an idle cluster, the only way to fix fragmentation is to reallocate the objects. Hence we require a defragmenter similar to kv-engine's.

      This will also require some logic on kv-engine side. They need to detect the right situation to invoke Magma's defragmenter. For example, when kv's defragmenter has looped over all vbuckets multiple times and fragmentation still isn't reducing.

      Note due to some unique characteristics of BlockCache, fragmentation problem is not as likely to arise in a running workload as it does for kv-engine, reason being:

      • Objects in BlockCache do not vary in size. So the same size classes get reused all the time.
      • It has a favourable allocation pattern such that for every block evicted, a new one takes its place. Hence memory is reused. An exception is when quota is readjusted and it causes of lot of deallocations in succession.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              srinath.duvuru Srinath Duvuru
              rohan.suri Rohan Suri
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty