Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-33056

request a streaming interface providing notification of changes to scopes and collections

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Duplicate
    • Major
    • None
    • 6.5.0
    • ns_server
    • None

    Description

      The query engine needs to maintain an up-to-date picture of the set of collections and scopes in each bucket. These are listed, along with various UID version information, in the manifest of each bucket. 
       
      The KV engine provides us a way to get the manifest and notifies us when we attempt to access a collection that is no longer present. Unfortunately these are a bit clumsy for maintaining an up-to-date picture of each bucket. We would need to poll the KV engine periodically to check whether the manifest has changed. Also, when the manifest does change, we would either have to redo the entire data structure that lists the scopes and collection, or diff the old and new manifest to allow for more specific changes. Doing this would definitely work, but is likely to become problematic at scale, in a heavily used system with hundreds of scopes each containing hundreds of collections.
       
      To that end we would like to be provided with a streaming interface that gives us more specific notification of collection and scope changes. Here are the features we would like to see:

      • Streaming interface of notifications of manifest changes
      • Per-bucket or per-cluster
      • Suggested operations:
        • Add scope
        • Drop scope
        • Add collection
        • Drop collection
      • Every notification contains original manifest UID and new manifest UID, so no changes are inadvertently skipped.

      We are open to a number of ways of providing this, either by augmenting an existing stream or creating a new one. Or perhaps Meta-KV would be the right way to do it.
       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              artem Artem Stemkovski
              johan.larson Johan Larson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty