Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-56879

Reduce overheads on low configuration nodes

    XMLWordPrintable

Details

    Description

      Indexer data pipeline overheads can be reduced for low configuration nodes. E.g.

      Consider a Capella configuration of 3 nodes with 16 Gig ram each, 48G total. With data + index co-located, the quota available is 80% i.e. 12.8GB. Index service gets 50% = 6.3GB. If data+index+query is co-located, index quota = 4.2GB.

      With such configuration, it is better to reduce allocations throughout the pipeline which are generally done to achieve throughput of the order of 500k ops/sec.

      Configs to consider:
      "indexer.mutation_manager.maxQueueMem"
      "indexer.settings.largeSnapshotThreshold"
      "indexer.settings.minVbQueueLength"
      "indexer.settings.sliceBufSize"
      "indexer.numSliceWriters"

      "projector.vbucketWorkers"
      "projector.dcp.numConnections"
      "projector.dcp.dataChanSize"

      "indexer.settings.build.batch_size" - By default 5, reduce it for low config nodes

      Use different configuation for slice during init vs incr build as incr build throughput is generally much lower than initial build:
      "indexer.settings.sliceBufSize"
      "indexer.numSliceWriters"

      This can either be done dynamically or Capella control plane can push the configuration based on pre-defined rules as all the settings are already configurable.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              devansh.srivastava Devansh Srivastava
              deepkaran.salooja Deepkaran Salooja
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty