Details
-
Task
-
Resolution: Fixed
-
Major
-
None
-
1
Description
While data load is ongoing, the state file creation is very aggressive. The state file number reaches to 4000+ in seconds. Checkpointing policy can be improved to add a minimum limit of at least few seconds to min.
magma_bench /data/single --kvstore 1 --ndocs 5200000 --nwrites 5200000 --nwriters 1 --batch-size 10 --memquota 10000000 --vallen 1024 --benchmark writeSequential --engine-config '
{"enable_lookup_on_set":true, "enable_directio":false, "enable_wal":true, "max_checkpoints":5}' --engine magma --stats stats.json --active-stats
Attachments
For Gerrit Dashboard: MB-48467 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
162034,1 | MB-48467 magma: Move checkpoint related functions to kvstore/checkpoint.cc | master | magma | Status: ABANDONED | 0 | 0 |
163485,6 | MB-48467 magma: Limit creation of checkpoints based on time | master | magma | Status: MERGED | +2 | +1 |
163492,6 | MB-48467 magma: Set default value of MinCheckpointCreationInterval | master | magma | Status: MERGED | +2 | +1 |
163750,7 | MB-48467: Add config for magma MinCheckpointCreationInterval | master | kv_engine | Status: MERGED | +2 | +1 |
163767,4 | MB-48467 magma: Ensure MinCheckpointCreationInterval <= CheckpointCreationInterval | master | magma | Status: MERGED | +2 | +1 |