Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-3799

moxi default hashing choice is not compatible with spy, enyim for memcached buckets (which use ketama_weighted)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.6.0 GA, 1.6.1, 1.6.4, 1.6.5, 1.6.5.2, 1.6.5.3, 1.6.5.4
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Security Level: Public
    • Flagged:
      Release Note

      Description

      Investigation of observed different caching between spymemcached and moxi in one environment shows that moxi is inadvertantly using the default from libmemcached. I say inadvertantly because that default may actually be the less common choice. It is different than what Enyim, Spymemcached and libketama do by default when selecting Ketama.

      libmemcached seems to use non-weighted Ketama, which seems to be specific to libmemcached and does not define the hash continuum the same way. Specifically, it seems to define the continuum with 100 points per server, rather than the 160 commonly used.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        ingenthr Matt Ingenthron created issue -
        Hide
        ingenthr Matt Ingenthron added a comment -

        I know it seems odd to file this as a low priority issue, and while it affects consistency when caching, it's believed that the number of times moxi is combined with Spy|Enyim is probably rare.

        Also, a fix is available, but disabled by default.

        Show
        ingenthr Matt Ingenthron added a comment - I know it seems odd to file this as a low priority issue, and while it affects consistency when caching, it's believed that the number of times moxi is combined with Spy|Enyim is probably rare. Also, a fix is available, but disabled by default.
        Hide
        ingenthr Matt Ingenthron added a comment -

        Fix is in MB-3798 and it's associated commit, but this bug should remain open since it really describes a different issue. Namely, this bug describes whether or not it should be turned on.

        Show
        ingenthr Matt Ingenthron added a comment - Fix is in MB-3798 and it's associated commit, but this bug should remain open since it really describes a different issue. Namely, this bug describes whether or not it should be turned on.
        Hide
        steve Steve Yen added a comment -

        The right choice seems to be to turn on ketama-weighted by default...

        ruby - fauna memcached (libmemcached wrapper) uses ketama-weighted

        :ketama_weighted => true,

        For PHP, the default distribution is MODULA distribution. However, from documentation in http://www.php.net/manual/en/memcached.constants.php, the safety tips around ketama hashing...

        Memcached::OPT_LIBKETAMA_COMPATIBLE

        Enables or disables compatibility with libketama-like behavior. When enabled, the item key hashing algorithm is set to MD5 and distribution is set to be weighted consistent hashing distribution. This is useful because other libketama-based clients (Python, Ruby, etc.) with the same server configuration will be able to access the keys transparently.

        Note:

        It is highly recommended to enable this option if you want to use consistent hashing, and it may be enabled by default in future releases.

        Show
        steve Steve Yen added a comment - The right choice seems to be to turn on ketama-weighted by default... ruby - fauna memcached (libmemcached wrapper) uses ketama-weighted :ketama_weighted => true, For PHP, the default distribution is MODULA distribution. However, from documentation in http://www.php.net/manual/en/memcached.constants.php , the safety tips around ketama hashing... Memcached::OPT_LIBKETAMA_COMPATIBLE Enables or disables compatibility with libketama-like behavior. When enabled, the item key hashing algorithm is set to MD5 and distribution is set to be weighted consistent hashing distribution. This is useful because other libketama-based clients (Python, Ruby, etc.) with the same server configuration will be able to access the keys transparently. Note: It is highly recommended to enable this option if you want to use consistent hashing, and it may be enabled by default in future releases.
        farshid Farshid Ghods (Inactive) made changes -
        Field Original Value New Value
        Fix Version/s 1.7 GA [ 10111 ]
        steve Steve Yen made changes -
        Priority Minor [ 4 ] Critical [ 2 ]
        farshid Farshid Ghods (Inactive) made changes -
        Assignee Steve Yen [ steve ] Frank Weigel [ frank ]
        Fix Version/s 1.7.1 [ 10183 ]
        Fix Version/s 1.7 GA [ 10111 ]
        Hide
        ingenthr Matt Ingenthron added a comment -

        I would argue against deferring this to 1.7.1. Why?

        This will be a change of compatibility when integrated. The way moxi hashes to memcached buckets will change. As pointed out previously, this is probably somewhat okay (given that it's a cache) during an upgrade, but the time to make the change is during a dot-minor version change, not a dot-micro patch update.

        Show
        ingenthr Matt Ingenthron added a comment - I would argue against deferring this to 1.7.1. Why? This will be a change of compatibility when integrated. The way moxi hashes to memcached buckets will change. As pointed out previously, this is probably somewhat okay (given that it's a cache) during an upgrade, but the time to make the change is during a dot-minor version change, not a dot-micro patch update.
        Hide
        steve Steve Yen added a comment -

        Pulling this back to 1.7.

        Show
        steve Steve Yen added a comment - Pulling this back to 1.7.
        steve Steve Yen made changes -
        Summary moxi hashing choice is not compatible with spy, enyim for memcached buckets moxi default hashing choice is not compatible with spy, enyim for memcached buckets (which use ketama_weighted)
        Assignee Frank Weigel [ frank ] Steve Yen [ steve ]
        Fix Version/s 1.7 Release Candidate [ 10182 ]
        Fix Version/s 1.7.1 [ 10183 ]
        Hide
        steve Steve Yen added a comment -
        Show
        steve Steve Yen added a comment - http://review.membase.org/6537
        steve Steve Yen made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.7.0-release-notes
        ingenthr Matt Ingenthron made changes -
        Fix Version/s 1.7 GA [ 10111 ]
        Fix Version/s 1.7 Release Candidate [ 10182 ]
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.7.0-release-notes 1.7.0-release-notes 1.7.1-release-notes

          People

          • Assignee:
            steve Steve Yen
            Reporter:
            ingenthr Matt Ingenthron
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes