DCP: Differentiate Between Delete and Expiry Operations

DCP doesn’t differentiate between delete/expiry – both come down the wire to dcp consumer (eventing) as opcode 0x58 (DCP_DELETION). Like differentiating b/w create & update (MB-26905) , we would need dcp producer to implement differentiation b/w expiry & delete as well. Delete caused due to Expiry and 'normal' Delete should also be captured separately.

Sample dump from a test(run by Abhishek Singh ): 

Created doc (DCP_MUTATION – opcode 0x57) with 10 sec expiry

2017-10-11T10:16:20.781+05:30 [Debug] DCPT[secidx:rawupr/0] ##54 DCP_SNAPSHOT for vb 84

2017-10-11T10:16:20.781+05:30 [Trace] dcp event dump &memcached.DcpEvent\{Opcode:0x57, Status:0x0, Datatype:0x1, VBucket:0x54, Opaque:0x54, VBuuid:0x1fe4b693c08b, Key:[]uint8\{0x74, 0x73, 0x5f, 0x30}, Value:[]uint8\{0x7b, 0x22, 0x74, 0x79, 0x70, 0x65, 0x22, 0x3a, 0x22, 0x74, 0x72, 0x61, 0x76, 0x65, 0x6c, 0x5f, 0x73, 0x61, 0x6d, 0x70, 0x6c, 0x65, 0x22, 0x2c, 0x22, 0x64, 0x6f, 0x63, 0x5f, 0x69, 0x64, 0x22, 0x3a, 0x30, 0x2c, 0x22, 0x73, 0x72, 0x63, 0x22, 0x3a, 0x22, 0x42, 0x4c, 0x52, 0x22, 0x2c, 0x22, 0x64, 0x73, 0x74, 0x22, 0x3a, 0x22, 0x44, 0x45, 0x4c, 0x22, 0x7d}, OldValue:[]uint8(nil), Cas:0x14ec6a9909510000, Seqno:0xc, RevSeqno:0x1, Flags:0x0, Expiry:0x59dda227, LockTime:0x0, Nru:0x2, SnapstartSeq:0x0, SnapendSeq:0x0, SnapshotType:0x0, FailoverLog:(*memcached.FailoverLog)(nil), Error:error(nil), Ctime:1507697180781464929}

2017-10-11T10:16:20.781+05:30 [Trace]      \{"type":"travel_sample","doc_id":0,"src":"BLR","dst":"DEL"}

 
Around 10 sec, got DCP_DELETION opcode instead of expected DCP_EXPIRATION (for this I made expiry pager to run every 1s instead of default 3600s)

2017-10-11T10:16:32.356+05:30 [Debug] DCPT[secidx:rawupr/0] ##54 DCP_SNAPSHOT for vb 84

2017-10-11T10:16:32.356+05:30 [Trace] dcp event dump &memcached.DcpEvent\{Opcode:0x58, Status:0x0, Datatype:0x0, VBucket:0x54, Opaque:0x54, VBuuid:0x1fe4b693c08b, Key:[]uint8\{0x74, 0x73, 0x5f, 0x30}, Value:[]uint8\{}, OldValue:[]uint8(nil), Cas:0x14ec6a9bbb400000, Seqno:0xd, RevSeqno:0x12, Flags:0x0, Expiry:0x0, LockTime:0x0, Nru:0x0, SnapstartSeq:0x0, SnapendSeq:0x0, SnapshotType:0x0, FailoverLog:(*memcached.FailoverLog)(nil), Error:error(nil), Ctime:1507697192356903614}

Eventing won’t be able to expose OnExpiry handler without DCP producer(KV-Engine) side changes. Because of above DCP behavior, if user creates doc timer to notify on expiry – they will also get call to their OnDelete handler as well when expiry pager runs. 

This is an issue as Delete and Expiry are two very different operations with vary different business implications.

From a consumer's(example: Eventing) perspective, it is very important that we provide a mechanism so that customers have OnDelete and OnExpiry handler. As of now, the only way possible is to create a Timer on the document few minutes before expiry. Though this might sound like a possible option, it has 2 consequences:

1. Timer's execution is not guaranteed at the specified time if the load on the system is high & the nodes are under provisioned.

2. Requires a metadata bucket(future collection) wherein the information is stored - this requires that Sizing be more considerate if there is non-trivial amount of documents and their respective timers.