Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-14950 Indexer memory usage grows higher than the index data size and is never freed.
  3. MB-15227

ForestDB does not release memory even after all inmemory snapshot handles are closed

    XMLWordPrintable

Details

    Description

      Ran the following test and I could reproduce the problem easily.

      1. Create 3 node cluster using cluster_run with one indexer node
      cluster_run -n 3
      cluster_connect -n 3 -T n0:index,n1:kv,n2:kv

      2. Create 4 indexes
      for i in

      {1..4}

      do
      cbindex -type create -index "first_name$i" -bucket default -fields=personal_details.first_name

      done

      3. Insert 10 million docs

      4. Drop the bucket to ensure that all indexes are deleted

      Observed that total indexer process memory remains ~6G, out of that 4.5G is consumed by forestdb.

      I have tried adding forestdb snapshot refcount stats in the indexer to verify that we are not holding any handle references.
      Also tried linking forestdb with jemalloc to ensure that it is not caused by system malloc not releasing memory to OS even after 2 hours.

      inmemory snapshot interval=200ms
      commit interval=30mins

      Memory growth is significantly less if you set commit interval to 3mins.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              sarath Sarath Lakshman
              sarath Sarath Lakshman
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty