Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47778

[BP 7.0.2] - XDCR - backfill replication spec reloading could misload spec from wrong data

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • No

    Description

      When XDCR is cold stopped and restarted, it reloads all metadata from metakv, such as replication spec. With 7.0, it also loads backfill replication specs.

      Backfill replication specs, and compressed backfill mapping files all live under the same "directory" under metakv. The current logic will try to load a spec only once by checking a specific prefix. If the prefix has been processed, it'll skip onto the next "backfill spec".

      There is a mistake in the reloading logic where the backfill replication service doesn't check if the loaded metakv doc is a backfill mapping doc instead of a backfill replication spec doc. This will cause the reloading mechanism to try to unmarshal the backfill mapping doc into the backfill replication spec, which will not return an error and thus fail silently. The backfill spec "reloaded" will have been empty and no tasks will be launched. 

      This will cause potential data loss (if goxdcr happens to be bounced/restarted)

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            neil.huang Neil Huang created issue -
            neil.huang Neil Huang made changes -
            Field Original Value New Value
            Link This issue Clones MB-47765 [ MB-47765 ]
            neil.huang Neil Huang made changes -
            Link This issue is a backport of MB-47765 [ MB-47765 ]
            neil.huang Neil Huang made changes -
            Link This issue Clones MB-47765 [ MB-47765 ]
            neil.huang Neil Huang made changes -
            Fix Version/s 7.0.1 [ 17104 ]
            Fix Version/s Neo [ 17615 ]
            Affects Version/s Neo [ 17615 ]
            wayne Wayne Siu made changes -
            Link This issue blocks MB-46308 [ MB-46308 ]
            wayne Wayne Siu made changes -
            Labels backport-candidate approved-for-7.0.1

            Build couchbase-server-7.0.1-5995 contains goxdcr commit 6d4d46d with commit message:
            MB-47778 - backfill spec service may not load backfill spec correctly when cold starting

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.1-5995 contains goxdcr commit 6d4d46d with commit message: MB-47778 - backfill spec service may not load backfill spec correctly when cold starting
            neil.huang Neil Huang made changes -
            Assignee Neil Huang [ neil.huang ] Pavithra Mahamani [ pavithra.mahamani ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.0.2 [ 18012 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.0.1 [ 17104 ]
            pavithra.mahamani Pavithra Mahamani made changes -
            Labels approved-for-7.0.1 approved-for-7.0.1 request-dev-verify
            pavithra.mahamani Pavithra Mahamani made changes -
            Assignee Pavithra Mahamani [ pavithra.mahamani ] Neil Huang [ neil.huang ]
            neil.huang Neil Huang made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            ianmccloy Ian McCloy made changes -
            Summary [BP 7.0.1] - XDCR - backfill replication spec reloading could misload spec from wrong data [BP 7.0.2] - XDCR - backfill replication spec reloading could misload spec from wrong data

            People

              neil.huang Neil Huang
              neil.huang Neil Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty