Details
-
Bug
-
Resolution: Won't Fix
-
Blocker
-
Cheshire-Cat
-
Untriaged
-
Unknown
Description
I believe the default number of shards has been increased from 4 to 32? Looking at ep_testsuite_basic tests.
This leaves a memory quota for magma at ~325KB…. 100MB Bucket Quota / 32 shards * 0.10 memory ratio = ~325KB
Magma can’t run in that small of a memory quota.
We know from recent archelogy done by Srinath and analysis by Dave R that couchstore really can’t either. Maybe in a test environment, but certainly not in any reasonable production environment. If couchstore ever tried to compact after a vbucket with few 1000 items, it would exceed the memory quota.
Magma shouldn’t crash as its doing today (working on tracking that down). But the question is, what should magma do when the memory quota < minimum needs?
I’m not sure what the actual minimal instance limit is for magma, but it’s at least 2.5MB which use to work.
Magma can’t just increase its memory usage past the memory quota it’s given because it’s going to record its memory usage and force ejections to memcache.
If the default # of shards is going to increase to 32, seems like the default bucket quota and the storage memory ratio should increase too.
If the old configuration was this…
100MB / 4 shards * 0.10 ratio = 2.5MB per magma instance
Maybe the new ratio should be
180MB / 32 shards * 0.50 ratio = 2.8MB per magma instance
This leaves the roughly the same ratio’s as before… 90MB to kv engine/memcache and ~2.8 MB per magma instance.