Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5279

user is unable to add the node back to the cluster because the failed over node is stuck in warmup phase

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.8.1-release-candidate
    • Fix Version/s: 1.8.1
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
    • Environment:
      CentOS 64 bit

      1.8.1-807

      Description

      Have not seen this issue previously ever on 181. Seems to be something new.

      Test case that failed
      swaprebalance.SwapRebalanceFailedTests.test_add_back_failed_node,replica=3,num-buckets=5,num-swap=2,swap-orchestrator=True,keys-count=800000

      1) Failover 2 nodes, including the orchestrator
      2) Rebalance them out
      3) Add them back and rebalance in, without cleaning them.

      [rebalance:error] [2012-05-10 8:48:59] [ns_1@10.1.3.74:<0.27930.16>:ns_rebalancer:do_wait_buckets_shutdown:170] Failed to wait deletion of some buckets on some nodes: [{'ns_1@10.1.3.77',
      {'EXIT',

      {old_buckets_shutdown_wait_failed, ["bucket-2", "bucket-1", "bucket-0", "bucket-4", "bucket-3"]}

      }}]

      [user:info] [2012-05-10 8:48:59] [ns_1@10.1.3.74:<0.27893.16>:ns_orchestrator:handle_info:246] Rebalance exited with reason {buckets_shutdown_wait_failed,
      [{'ns_1@10.1.3.77',
      {'EXIT',

      {old_buckets_shutdown_wait_failed, ["bucket-2","bucket-1","bucket-0", "bucket-4","bucket-3"]}

      }}]}

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        karan Karan Kumar (Inactive) created issue -
        karan Karan Kumar (Inactive) made changes -
        Field Original Value New Value
        Fix Version/s 1.8.1 [ 10295 ]
        Affects Version/s 1.8.1-release-candidate [ 10299 ]
        Component/s ns_server [ 10019 ]
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Frank Weigel [ frank ]
        dipti Dipti Borkar made changes -
        Labels customer
        Assignee Frank Weigel [ frank ] Aleksey Kondratenko [ alkondratenko ]
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Dipti Borkar [ dipti ]
        farshid Farshid Ghods (Inactive) made changes -
        Summary Rebalance failed due to buckets_shutdown_wait_failed user is unable to add the node back to the cluster because the failed over node is stuck in warmup phase
        dipti Dipti Borkar made changes -
        Assignee Dipti Borkar [ dipti ] Chiyoung Seo [ chiyoung ]
        dipti Dipti Borkar made changes -
        Sprint Status Current Sprint
        dipti Dipti Borkar made changes -
        Sprint Priority 0
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Chiyoung Seo [ chiyoung ] Aleksey Kondratenko [ alkondratenko ]
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        dipti Dipti Borkar made changes -
        Sprint Status Current Sprint
        Sprint Priority 0
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            alkondratenko Aleksey Kondratenko (Inactive)
            Reporter:
            karan Karan Kumar (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: