Uploaded image for project: 'Couchbase Documentation'
  1. Couchbase Documentation
  2. DOC-8556

XDCR - Concepts of Collections Migration mode and Deletes

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Unresolved
    • Cheshire-Cat
    • Cheshire-Cat
    • xdcr
    • DOC-2021-May23-S10, DOC-2021-Jun06-S11, DOC-2021-Jun20-S12
    • 1

    Description

      MB-46359 brought an interesting point where when an advanced filtering is used, any deletes issued are passed through the filter without checking against the filter. This is because the deletes being sent from KV do not contain document body. (See MB-46359 for info and comments on the reason behind this decision)

      With collections migration, this same concept is now applied in multiple pathways: a single source default collection can now replicate to multiple target collections, each with a specific filtering expression. Thus, a single delete from the source default collection will be replicated to all mapped target collections under migration mode, regardless of the specified expression.

      The important fact is that it is quite easy for an end-user to forget that deletes are passed through a filter. Especially now with multiple filtering expressions for migration rule, this means that a single document deleted from the source default collection will affect (pass the deletion to) all mapped target collections.

      Customers need to be aware and reminded of this fact when they are reading on how to set up migration, as being oblivious to this behavior can lead to data loss if the customer mistakenly thinks the deletes are also filtered.

      A good general and conservative recommendation would be:

      Do not delete source default collections documents until all migrations are finished, since a single source default collection document delete will be propagated to all mapped target collections under migration mode.

      It may be useful per MB-46359 to remind the customers that XDCR does provide a global "Do not replicate delete/expiry" option at the filter level, which applies to collections migration. The caveat is that of course, if deletes occur on the source, then the document count between the source default collection and all target collections will no longer match.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          tony.hillman Tony Hillman added a comment - - edited

          Partially handled under DOC-8555. This ticket will track changes to the migration step by step procedure as appropriate.

          tony.hillman Tony Hillman added a comment - - edited Partially handled under DOC-8555 . This ticket will track changes to the migration step by step procedure as appropriate.
          tony.hillman Tony Hillman added a comment - PR at PR at  https://github.com/couchbase/docs-server/pull/1958
          tony.hillman Tony Hillman added a comment -

          Approved and merged.

          tony.hillman Tony Hillman added a comment - Approved and merged.

          People

            tony.hillman Tony Hillman
            neil.huang Neil Huang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty