Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29132

indexer.numSliceWriters=runtime.NumCPU() policy doesn't work very well when there are many Plasma instances

    XMLWordPrintable

Details

    Description

      Build 5.5.0-2414.

      Mapping the number of writer threads to the number of CPU cores is not always a good idea. I attempted to create ten partitioned indexes on my 48-core server and the indexer process was OOM-killed almost immediately.

      Although it's possible to mitigate this issue by tuning indexer.numSliceWriters option, a global thread pool with fair scheduling seems more appropriate.

      MB-29009 describes the environment and secondary indexes.

      Attachments

        1. indexer_5.5.0-2785_c1.svg
          236 kB
        2. indexer_5.5.0-2785_c2.svg
          218 kB
        3. indexer_5.5.0-2785_c3.svg
          225 kB
        4. indexer_5.5.0-2785_c4.svg
          216 kB
        5. indexer_5.5.0-2785-non_part.svg
          244 kB
        6. indexer_rss.png
          indexer_rss.png
          29 kB

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-29137 Slow initial indexing of multiple partitioned indexes

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Task - A task that needs to be done. MB-29909 Plasma: Evaluate the wasteful work performed by evictors due to conflicts with writers during DGM

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. MB-29818 [plasma] Make number of evictors configurable

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              sarath Sarath Lakshman
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty