Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36901

Increase bucket quota for magma ep engine tests

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      I believe the default number of shards has been increased from 4 to 32? Looking at ep_testsuite_basic tests.

      This leaves a memory quota for magma at ~325KB….  100MB Bucket Quota / 32 shards * 0.10 memory ratio = ~325KB

      Magma can’t run in that small of a memory quota.

      We know from recent archelogy done by Srinath and analysis by Dave R that couchstore really can’t either. Maybe in a test environment, but certainly not in any reasonable production environment. If couchstore ever tried to compact after a vbucket with few 1000 items, it would exceed the memory quota.

      Magma shouldn’t crash as its doing today (working on tracking that down). But the question is, what should magma do when the memory quota < minimum needs?

      I’m not sure what the actual minimal instance limit is for magma, but it’s at least 2.5MB which use to work.

      Magma can’t just increase its memory usage past the memory quota it’s given because it’s going to record its memory usage and force ejections to memcache.

      If the default # of shards is going to increase to 32, seems like the default bucket quota and the storage memory ratio should increase too.

      If the old configuration was this…

      100MB / 4 shards * 0.10 ratio = 2.5MB per magma instance

      Maybe the new ratio should be

      180MB / 32 shards * 0.50 ratio = 2.8MB per magma instance

      This leaves the roughly the same ratio’s as before… 90MB to kv engine/memcache and ~2.8 MB per magma instance.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            scott.lashley Scott Lashley
            scott.lashley Scott Lashley
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty