Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-5378

Rebalance failes because of memcached request timeout

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Major
    • 2.0
    • 1.8.1
    • couchbase-bucket, ns_server
    • Security Level: Public
    • None
    • 7 Node centos cluster
      1 Bucket, 64 vBuckets

    Description

      Setup [cluster : 10.1.3.74 to 0.1.3.80]

      1. Setup a 7 node cluster [74, 75, 76, 77, 78,79 ,80]
      2. Failovered nodes [75, 77, 78] - successful failovers for all
      3. Rebalance cluster [ejected Nodes - 75,77,78 cluster - 74, 75, 76, 77, 78, 79, 80]

      Output
      Rebalance Failed as
      [2012-05-24 07:04:43,473] - [rest_client:749] ERROR -

      {u'status': u'none', u'errorMessage': u'Rebalance failed. See logs for detailed reason. You can try rebalance again.'}

      - rebalance failed
      ERROR]

      This is an automated test-_add_back_failed_node on Swap RebalanceTests.
      http://qa.hq.northscale.net/view/1.8.1/job/181-centos-64-swaprebalance-tests/lastBuild/testReport/junit/swaprebalance/SwapRebalanceFailedTests/swaprebalance_SwapRebalanceFailedTests_test_add_back_failed_node_get_logs_True_replica_1_num_swap_3_num_nodes_7_num_buckets_1_spec_py_swaprebalance/

      Error from the ns_server log
      [error_logger:error] [2012-05-24 13:09:15] [ns_1@10.1.3.79:error_logger:ale_error_logger_handler:log_report:72]
      =========================CRASH REPORT=========================
      crasher:
      initial call: ebucketmigrator_srv:init/1
      pid: <0.884.0>
      registered_name: []
      exception exit: downstream_closed
      in function gen_server:terminate/6
      ancestors: ['ns_vbm_sup-default','single_bucket_sup-default',<0.775.0>]
      messages: [

      {'EXIT',<0.891.0>,killed}

      ]
      links: [<0.786.0>,<0.57.0>]
      dictionary: []
      trap_exit: true
      status: running
      heap_size: 2584
      stack_size: 24
      reductions: 13663
      neighbours:

      Attaching the ns_server logs at https://s3.amazonaws.com/bugdb/jira/swap_check/swap_01.tar

      Attachments

        1. 10.3.121.182-8091-diag.txt.gz
          9.13 MB
        2. 10.3.121.183-8091-diag.txt.gz
          5.06 MB
        3. 10.3.121.185-8091-diag.txt.gz
          11.15 MB
        4. 10.3.2.10-8091-diag.txt.gz
          12.04 MB
        5. 10.3.2.11-8091-diag.txt.gz
          14.70 MB
        6. 10.3.2.12-8091-diag.txt.gz
          12.27 MB
        7. 10.3.2.16-8091-diag.txt.gz
          12.23 MB
        8. 10.3.2.75-8091-diag.txt.gz
          12.22 MB
        9. 10.3.2.76-8091-diag.txt.gz
          12.27 MB
        10. 10.3.2.77-8091-diag.txt.gz
          12.30 MB
        11. 10.3.2.78-8091-diag.txt.gz
          12.25 MB
        12. 10.3.2.79-8091-diag.txt.gz
          12.31 MB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ketaki Ketaki Gangal (Inactive)
            ketaki Ketaki Gangal (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty