Uploaded image for project: 'Couchbase C client library libcouchbase'
  1. Couchbase C client library libcouchbase
  2. CCBC-119

Feed a different string (or hash?) for vBucket attribution

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0beta2
    • Fix Version/s: 2.0.0
    • Component/s: library
    • Security Level: Public
    • Labels:
      None

      Description

      We currently are in a situation where we would like to use a different string than the key for hashing and selection of the destination vBucket. This can be useful to keep all of a single user data on many keys but on the same vBucket (and therefore servers), and allow for node downtime to only affect a percentage of the user base instead of the full user base. More generally, it could be used to gain better control over the distribution of data over the nodes.

      Looking at the current code, I guess it would be safer not to touch the hashing but just allow for the hashing of a separate string?

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        trond Trond Norbye added a comment -

        To facilitate this we need to create an extended version of our arguments (the lcb_get_cmd_t etc) that includes a "blob" to use for hashing instead of the key.

        We need to verify that the code in libcouchbase that handles "not-my-vbuckets" doesn't rehash the key, but just use the updated servermap to locate the correct server.

        Show
        trond Trond Norbye added a comment - To facilitate this we need to create an extended version of our arguments (the lcb_get_cmd_t etc) that includes a "blob" to use for hashing instead of the key. We need to verify that the code in libcouchbase that handles "not-my-vbuckets" doesn't rehash the key, but just use the updated servermap to locate the correct server.
        Hide
        avsej Sergey Avseyev added a comment -

        I thought we removed *_by_key() functions for reason here http://review.couchbase.org/20073

        See Matt's comment here http://review.couchbase.org/14956

        > At this point in time, I do not think we want to include this. It is
        > accurate that python exposes this to the user, but that's more out
        > of python's legacy as testing code than a decision to add that to
        > API.
        >
        > vbuckets are generally to be abstracted away from the user's app.
        > let's not leak the abstraction out any more than we need to.

        Show
        avsej Sergey Avseyev added a comment - I thought we removed *_by_key() functions for reason here http://review.couchbase.org/20073 See Matt's comment here http://review.couchbase.org/14956 > At this point in time, I do not think we want to include this. It is > accurate that python exposes this to the user, but that's more out > of python's legacy as testing code than a decision to add that to > API. > > vbuckets are generally to be abstracted away from the user's app. > let's not leak the abstraction out any more than we need to.
        Hide
        ingenthr Matt Ingenthron added a comment -

        Given that there is a user requirement for it, perhaps we should re-add it. Let me check into it.

        The concern in the past has been that this will likely cause unbalanced clusters and can create confusion on how one can get to the data from one client that isn't available from another client.

        Show
        ingenthr Matt Ingenthron added a comment - Given that there is a user requirement for it, perhaps we should re-add it. Let me check into it. The concern in the past has been that this will likely cause unbalanced clusters and can create confusion on how one can get to the data from one client that isn't available from another client.
        Hide
        mnunberg Mark Nunberg added a comment -

        Would it not be simpler from our perspective to simply provide a 'hash' field in the request structure?

        Much less boilerplate in our code - change would simply be to use an existing hash if it's already there, and then we can do the normal vbucket mapping If it's 0, then calculate it.

        This is actually quite familiar with APIs that do hashing.

        Maybe expose an additional lcb_hash_key function.

        Show
        mnunberg Mark Nunberg added a comment - Would it not be simpler from our perspective to simply provide a 'hash' field in the request structure? Much less boilerplate in our code - change would simply be to use an existing hash if it's already there, and then we can do the normal vbucket mapping If it's 0, then calculate it. This is actually quite familiar with APIs that do hashing. Maybe expose an additional lcb_hash_key function.
        Hide
        trond Trond Norbye added a comment -

        http://review.couchbase.org/#/c/22395/

        Mark: unfortunately that would require a bigger change in the current code (we would have to supply a new API from libvbucket). Given that we haven't released any version of libcouchbase 2.0, I decided that we can just put this into the basic struct..

        Show
        trond Trond Norbye added a comment - http://review.couchbase.org/#/c/22395/ Mark: unfortunately that would require a bigger change in the current code (we would have to supply a new API from libvbucket). Given that we haven't released any version of libcouchbase 2.0, I decided that we can just put this into the basic struct..

          People

          • Assignee:
            trond Trond Norbye
            Reporter:
            stelcheck Marc Trudel
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes