Details
-
Improvement
-
Resolution: Unresolved
-
Major
-
7.2.0
-
7.2.0-5230
-
0
Description
Steps To Recreate:
Steps to Recreate:
- Create a 3 node cluster
- Create a magma buckets with (bucket_history_retention_seconds=259200,bucket_history_retention_bytes=10000000000000,) (vbuckets = 1024, replicas = 1)
- Start data loading,
- Rebalance in one node
- When disk usage is around 32 Tb, start rebalancing out one node
- Pause rebalance and change bucket_history_retention_seconds=300
- Waited long enough (for more than 30 mins, infact it was same after 5 hours), and observed oldest document in magma history is 2 days old(calculated thru CAS)(though start seq number got moved as expected, becz of changes bucket_history_retention_seconds)(checked for vb_0)
- 32TB of data with one replica. (2048 vbcukets) so each vbucket size is 15GB
- sstable will be 5% of that so 750MB, sstable was not hitting 750MB, so holding on to longer duration history
- Had a discussion with Apaar, and showed the cluster to him, based on the discussion logging an improvement bug.
Logs:
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1678605553/collectinfo-2023-03-12T071915-ns_1%40172.23.110.69.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1678605553/collectinfo-2023-03-12T071915-ns_1%40172.23.110.65.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1678605553/collectinfo-2023-03-12T071915-ns_1%40172.23.110.64.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1678605553/collectinfo-2023-03-12T071915-ns_1%40172.23.110.67.zip
https://cb-engineering.s3.amazonaws.com/seq_history_300_Second_3.txt
https://cb-engineering.s3.amazonaws.com/treedata_300.txt
Attachments
Issue Links
- relates to
-
MB-55861 CDC: History retained on disk should exclude items from collections with history=false
-
- Resolved
-