Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
5.0.0
-
Untriaged
-
Yes
Description
cbXDCR test failed when memcached connection had xattr feature enabled.
The test scenario is as follows:
- bi-directional replication between spock clusters with two nodes each, denoted as A and B
- insert, update, and delete documents on cluster A.
- remove one node from cluster A. This gets replications in both directions restarted. Restarted replication would re-replicate mutations.
Case 1: When the memcached connection for data replication did not have xattr enabled, the test ran fine. After replications were restarted in step 3, the insertion/update/deletion were replicated to B as expected. Cluster A and B were in sync after a short while.
Case 2: When the memcached connection for data replication had xattr enabled, after replications were restarted in step 3, both replications were getting KEY_ENOENT errors from setMeta requests. It seems that the "target" memcached still saved the setMeta mutations, and the setMeta mutations were considered new mutations and replicated to the other cluster. This formed an infinite loop and the number of mutations processed by XDCR was way more than the original mutations performed by test script.
In my experiment, I simplified the scenario by not requesting xattr from dcp. Afterward the setMeta requests in case 1 and case 2 were exactly the same, all having datatype of 0. The same issue persisted. The enabling of xattr feature through helo command is the only difference between case 1 and case 2 and has to be the cause of the problem.
cbcollect files from both clusters (since both clusters are both the source and the target cluster) are attached. The first two files with "n_2" and "n_3" in the file names are from cluster B. The third file with "n_1" in the file name is from cluster A.
Attachments
Issue Links
- relates to
-
MB-23618 KEY_ENOENT error from setMeta request with XATTR feature enabled
- Closed