Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-30901

Differentiate Between metadata and document body update

    XMLWordPrintable

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • DCP
    • None

    Description

      DCP should have the semantics by which consumers of DCP can differentiate between a possible change that affected only the metadata(for ex: xattrs) vs that which affects the document body.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            drigby Dave Rigby added a comment -

            Couple of clarifying questions:

            1. What are your expectations for DCP backfill - Do you require that consumers can differentiate between metadata and document body updates when reading historic data from disk?
            2. What fields do you definite as metadata?
            3. Do you need to distinguish between different metadata fields - i.e. a mutation which only changed TTL vs. one which changed user XATTRs vs. one which changed system XATTRs?

            It might also be helpful if you could expand the description to describe what your use-case(s) are, and why you require this.

            drigby Dave Rigby added a comment - Couple of clarifying questions: What are your expectations for DCP backfill - Do you require that consumers can differentiate between metadata and document body updates when reading historic data from disk? What fields do you definite as metadata? Do you need to distinguish between different metadata fields - i.e. a mutation which only changed TTL vs. one which changed user XATTRs vs. one which changed system XATTRs? It might also be helpful if you could expand the description to describe what your use-case(s) are, and why you require this.

            Venkatraman Subramanian quick ping on DaveR's questions above. Additionally, what is the priority of this enhancement versus other DCP requests for Mad Hatter?

            dhaikney David Haikney added a comment - Venkatraman Subramanian quick ping on DaveR's questions above. Additionally, what is the priority of this enhancement versus other DCP requests for Mad Hatter?

            #1 : when reading from disk(and assuming DCP knows if it is reading from disk or memory), i would guess that we need special case it, and call it as Document Update(even when it is just a metadata update).

            #2 : As of now - all meta.* and system-xattrs and user-axattrs. (but read onto #3)

            #3 : good question. Essentially this is a corollary to #2. If we can differentiate between the 3(meta, user-xattr and system-xattr), it will be good. But a P0, would be just differentiate between meta(both meta.* and all types of xattrs) and body updates. (Tagging Matt Ingenthron for any thoughts on this)

            Priya Rajagopal / Adam Fraser : can you add some more details w.r.t SG here.

            The background to this MB is essentially that SG updates the xattrs(system) and this causes mutations on the DCP. For Eventing, this is a problem, since for non-idempotent operations in Eventing will lead to confusion as multiple updates are seen by Eventing. Eventing being a handle to consume the events, this requirement becomes critical.

            Also, I believe that this request can be satisfied only for new events, and not for historic events.

            venkatraman.subramanian Venkatraman Subramanian (Inactive) added a comment - #1 : when reading from disk(and assuming DCP knows if it is reading from disk or memory), i would guess that we need special case it, and call it as Document Update(even when it is just a metadata update). #2 : As of now - all meta.* and system-xattrs and user-axattrs. (but read onto #3) #3 : good question. Essentially this is a corollary to #2. If we can differentiate between the 3(meta, user-xattr and system-xattr), it will be good. But a P0, would be just differentiate between meta(both meta.* and all types of xattrs) and body updates. (Tagging Matt Ingenthron for any thoughts on this) Priya Rajagopal / Adam Fraser : can you add some more details w.r.t SG here. The background to this MB is essentially that SG updates the xattrs(system) and this causes mutations on the DCP. For Eventing, this is a problem, since for non-idempotent operations in Eventing will lead to confusion as multiple updates are seen by Eventing. Eventing being a handle to consume the events, this requirement becomes critical. Also, I believe that this request can be satisfied only for new events, and not for historic events.
            drigby Dave Rigby added a comment -

            Assigning to Jim as he's scoping the proposed DCP changes for Mad-Hatter.

            drigby Dave Rigby added a comment - Assigning to Jim as he's scoping the proposed DCP changes for Mad-Hatter.
            asingh Abhishek Singh (Inactive) added a comment - Another user asking for it https://forums.couchbase.com/t/eventing-function-fires-twice-because-of-sync-gateway/18985

            People

              jwalker Jim Walker
              venkatraman.subramanian Venkatraman Subramanian (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty