Details
-
Improvement
-
Resolution: Fixed
-
Major
-
4.0.0-dp.1
-
None
Description
In 4.x, the connector will support rollback mitigation: events will not be written to Elasticsearch until they have been persisted to all replicas. This will prevent rollbacks from affecting the data in Elastic search, in all but the most extreme multiple-failure scenarios.
In addition, when the connector saves DCP stream checkpoint information, it will include the vbucket UUID (as well as sequence number) so that any rollbacks that do occur may be detected. Rollback mitigation will prevent most rollbacks caused by errors, but we still need to handle some "normal" rollback conditions like when a bucket is flushed, for example.
In response to a rollback, the connector will restart the DCP stream from the rollback sequence number. However, due to technical limitations, the connector is unable to revert documents in Elasticsearch to their pre-rollback state. (This is why rollback mitigation is so important.)