Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7571

online upgrade 2.0.0 -> 2.0.1 rebalance exited with {unexpected_exit, {'EXIT',<0.2421.1>, {{badmatch, [{'EXIT', {timeout, {gen_server,call, [<0.2412.1>,had_backfill,30000]}}}]}, [{ns_single_vbucket_mover, '-wait_backfill_determination/1-fun-1-',1}]}}}

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Critical
    • 2.0.1
    • 2.0.1
    • installer, ns_server
    • Security Level: Public
    • None
    • centos5.7 with 4GB of RAM
    • PCI Team - Sprint 1

    Description

      steps:
      1. 3 1976 nodes in cluster with 3 different buckets, 1 view in each bucket, start upload data for each bucket through mcsoda:
      ip:10.3.121.112
      ip:10.3.121.114
      ip:10.3.121.115

      2. add 2 139build nodes:10.3.121.116, 10.3.121.117 and remove 10.3.121.114,10.3.121.115 ( swap rebalance)

      rebalance exit with progress ~ 40% when each bucket contains only ~400K at the time

      <0.2363.1> exited with {unexpected_exit,
      {'EXIT',<0.2421.1>,
      {{badmatch,
      [{'EXIT',
      {timeout,

      {gen_server,call, [<0.2412.1>,had_backfill,30000]}

      }}]},
      [

      {ns_single_vbucket_mover, '-wait_backfill_determination/1-fun-1-',1}

      ]}}}

      also please note, that error message doesn't contain any mention of rebalance ( <0.2363.1> exited with {unexpected_exit)

      I also noticed that during the rebalance stats graphics do not show up on, and statistics for some buckets did not change at all. for example, the number of operations per second for default bucket was always 526 operations per second.
      only when rebalance failed and I stopped mcsoda loading all stats were updated

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            farshid Farshid Ghods (Inactive)
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty