Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59289

Prevent creation of new ActiveStream if bucket quota will be exceeded

    XMLWordPrintable

Details

    • 0
    • Yes
    • KV 2023-4

    Description

      Currently, we have no checks if the bucket quota will be exceeded when a new ActiveStream is created. This can cause memory usage to exceed the bucket quota in certain situations. 

      For example, I created a single node cluster with 1GB of memory usage and 999 collections. Then I ran dcpdrain with a stream config that maps each collection to a unique ActiveStream (mimicing CBAS). This creates 999 * 1024 = 1,022,976 ActiveStreams in total and takes up a total of ~1GB of bucket quota just to manage all those connections. Because no check is included to validate if the bucket quota will be exceeded when a new ActiveStream is created, the total memory used increases above the bucket quota. As seen in my testing attached.

      This issue is noticeable in small bucket quota environments intending to create a large number of ActiveStreams. Therefore, to prevent exceeding the Bucket quota we should add a sanity check to make sure the bucket quota will not be exceeded when creating a new ActiveStream.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavlos.georgiou Pavlos Georgiou
              mohammad.zaeem Mohammad Zaeem
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty