Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5049

We have evidence of node continuing rebalance tens of minutes after disconnect happened

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.8.1
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
      None

      Description

      In one of customers (on EC2) we had some strange case as noted in SUBJ. Quite possibly there was some serious lack of CPU for VM, but let's at least inspect code for other possible causes of this problem.

      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-5049=message:MB-5049

        Activity

        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-ns-server-2-0 #339 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/339/)
        set keepalive on ebucketmigrator sockets. MB-5049 (Revision 57455f88120465bc89ee4cad14c09abe97fe8688)

        Result = SUCCESS
        Aliaksey Kandratsenka :
        Files :

        • src/ebucketmigrator_srv.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #339 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/339/ ) set keepalive on ebucketmigrator sockets. MB-5049 (Revision 57455f88120465bc89ee4cad14c09abe97fe8688) Result = SUCCESS Aliaksey Kandratsenka : Files : src/ebucketmigrator_srv.erl
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        While it's possible to have some nodes down and rebalance continuing with current code, I've found no way how complete disconnect of master node can not fail rebalance. Closing. And I think this customer's case is something about environment, possibly EC2 just took CPU away from them for some reason

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - While it's possible to have some nodes down and rebalance continuing with current code, I've found no way how complete disconnect of master node can not fail rebalance. Closing. And I think this customer's case is something about environment, possibly EC2 just took CPU away from them for some reason
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-ns-server-2-0 #342 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/342/)
        fail rebalance asap if any of nodes goes down. MB-5049 (Revision b945a08f60d6e16df848e803053ebb93021601be)

        Result = SUCCESS
        Aliaksey Kandratsenka :
        Files :

        • src/ns_vbucket_mover.erl
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #342 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/342/ ) fail rebalance asap if any of nodes goes down. MB-5049 (Revision b945a08f60d6e16df848e803053ebb93021601be) Result = SUCCESS Aliaksey Kandratsenka : Files : src/ns_vbucket_mover.erl

          People

          • Assignee:
            alkondratenko Aleksey Kondratenko (Inactive)
            Reporter:
            alkondratenko Aleksey Kondratenko (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes