Details
-
Improvement
-
Resolution: Won't Do
-
Major
-
Morpheus
-
None
Description
As discussed in the design document and various meetings, in order to be able to know the collection manifest at a given point in time we need to be able to get collection events. To make this efficient they will be stored in the footer of SSTables and then cleaned up during compaction, as they are only needed for the CDC period.
Magma does not know what a collection, and therefore what a collection document, is. To know what documents to stash in the header KV will provide a callback which, given a document, will return true if the document should be stashed. This should be invoked when a memtable is flushed.