Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-55861

CDC: History retained on disk should exclude items from collections with history=false

    XMLWordPrintable

Details

    • 0
    • Magma June-16 2023, Magma July-27 2023

    Description

      In the current 7.2 CDC functionality, the on-disk history retention "window" keeps every version written by KV upto the configured retention limits (i.e. size and/or time).

      We could envisage a bucket configuration where there is only 1 collection which actually enables history and as such never deduplicates. However if there are other collections configured (with history=false), and those other collections are dominating the workload the history retention window can become dominated/consumed by those non-history collections.

      E.g. If the user thinks they have 15 minutes of history (capped at 1GB), they may not actually have the last 15 minutes of their history collection.

      We've discussed this issue before but never committed to solving it in 7.2 unless time allowed, but allowing the history retention window to be "partially" compacted (removal of non-historical collections) could be very useful to the overall feature.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              apaar.gupta Apaar Gupta
              jwalker Jim Walker
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty