Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-20623

Warmup via the access log using a huge amount of memory and causes a low resident ratio

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 4.6.0
    • 4.1.1, 4.5.0
    • couchbase-bucket
    • Security Level: Public
    • Untriaged
    • No

    Description

      Warmup via the Access log keeps a huge about of temporary data structures around during the whole warmup process, as a result of this it cause the warmup process to finish early as it reaches the low watermark sooner. Which in turn means the resident ratio is also low.

      There are three data structures created in memory when using the access.log:

      1. A list of keys in the access log and their sequence number at the time the access log was generated - MutationLogHarvester::committed
      2. Another list of keys and their updated sequence number - MutationLogHarvester::apply::fetches
      3. A temporary back ground fetch item for every key in the list above - batchWarmupCallback::items2fetch

      The crux of this is the batched warmup (CBD-440), which doesn't have an upper limit on the batch size. Therefore if you have (for example) 6M items in a single vBucket, then Couchbase Server will actually create of the order of 3x 6M == 18M temporary items.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-20623
          # Subject Branch Project Status CR V

          Activity

            People

              drigby Dave Rigby (Inactive)
              pvarley Patrick Varley (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty