Uploaded image for project: 'Couchbase Documentation'
  1. Couchbase Documentation
  2. DOC-11141

XDCR - binary document replication behavior and setting

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • Morpheus
    • None
    • xdcr
    • None
    • 0

    Description

      See parent ticket of MB-56739 for new improvements and related CBSE where the lack of documentation on this subject caused confusion.

      We should at least document that before MB-56739:

      1. The default behavior is that binary documents are replicated to the target.
      2. If a filter expression contains key filter expression only (https://github.com/couchbase/goxdcr/blob/6a5234c23dc30231c09bd93c6aa32987984fcc3a/base/simple_utils.go#L1451) or if it contains xattr only, or a combination of both key and xattr alone, then the binary document will be processed accordingly to the filter expression. (https://github.com/couchbase/goxdcr/blob/6a5234c23dc30231c09bd93c6aa32987984fcc3a/base/filter/filterUtils.go#L109)
      3. The converse of the above is that if a filter expression contains a non-key or non-xattr expression (i.e. refers to the document body), then the binary document is forcefully passed through the filter and replicated.

      With MB-56739, we now have a replication setting of filterBinary which defaults to false. This is not in the UI and only available in the REST API as of now.
      If this is set to true, then this will override all currently expected behavior, including all 3 above. In other words, if this is set to true, then binary documents are not replicated under all circumstances, even in the absence of a specified filter expression.

      Secondly, we should add the stats that are added alongside this MB, if this is within scope.

      Prometheus will have a new counter called

      # HELP xdcr_binary_filtered_total Number of documents filtered that were binary documents.
      # TYPE xdcr_binary_filtered_total counter
      xdcr_binary_filtered_total {targetClusterUUID="21e49c6b6f6598557b7a2ab6cbbae2fb", sourceBucketName="B1", targetBucketName="B2", pipelineType="Backfill"} 0
      xdcr_binary_filtered_total {targetClusterUUID="21e49c6b6f6598557b7a2ab6cbbae2fb", sourceBucketName="B1", targetBucketName="B2", pipelineType="Main"} 1
      

      Which captures the number of mutations that are not replicated due to this setting being in effect.

      Attachments

        Issue Links

          Activity

            People

              tony.hillman Tony Hillman (Inactive)
              neil.huang Neil Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty