Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34678

Forestdb does not calculate max memory correctly in containerized environments

    XMLWordPrintable

Details

    • Untriaged
    • No

    Description

      cbbackupmgr has code which automatically sets the forestdb buffer cache to 2GB.

      If the error FDB_RESULT_TOO_BIG_BUFFER_CACHE is returned from forestdb then it will continue to halve the buffer size until the error is not returned.

      Forestdb will only return this error if the memory size of the machine (as forestdb sees it) is less than the specified buffer size - https://github.com/couchbase/forestdb/blob/8642f9b68c7baf5ad02d58be878bb154b2e6bb2b/src/forestdb.cc#L2125-L2128.

      The code it uses to do this is the sysctl call to HW_MEMSIZE, which returns the amount of physical memory on the system - https://github.com/couchbase/forestdb/blob/8642f9b68c7baf5ad02d58be878bb154b2e6bb2b/utils/system_resource_stats.cc#L46-L53.

      The problem is in a containerized environment, the container will see all of the machines memory, even if a user has chosen to limit the memory usage.
      The memory limiting is instead done via cgroups, which you have to check the limit of in a different way, see also MB-24946 where a change has had to be made to ns_server to correctly detect the available memory in containerized environments.

      The common effect of this issue is that it's impossible to run cbbackupmgr in a containerized environment with less than 2GB of RAM (ideally with a little extra for overhead).
      If we correctly detected the memory available within forestdb then the auto-adjustment of the buffer size would be able to kick in and would allow cbbackupmgr to run in containers with less than 2GB RAM.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            srinath.duvuru Srinath Duvuru
            matt.carabine Matt Carabine (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty