Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-9691

rebalance repeated failed when add nodes back into cluster

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Blocker
    • 2.5.0
    • 2.5.0
    • ns_server
    • Security Level: Public
    • None
    • ubuntu 10.04 64-bit, ubuntu 12.04 64-bit
    • Ubuntu 64-bit

    Description

      Environment: 12 ubuntu servers with 4 GB RAM
      Group 1
      10.3.3.61
      DISTRIB_RELEASE=10.04
      10.3.3.74
      DISTRIB_RELEASE=10.04
      10.3.3.67
      DISTRIB_RELEASE=10.04

      Group 2
      10.3.3.63
      DISTRIB_RELEASE=10.04
      10.3.4.219
      DISTRIB_RELEASE=10.04
      10.3.4.220
      DISTRIB_RELEASE=10.04

      Group 3
      10.3.3.75
      DISTRIB_RELEASE=10.04
      10.3.3.71
      DISTRIB_RELEASE=10.04
      10.1.2.85
      DISTRIB_RELEASE=12.04

      Group 4
      10.3.3.58
      DISTRIB_RELEASE=10.04
      10.3.3.64
      DISTRIB_RELEASE=10.04
      172.23.105.228
      DISTRIB_RELEASE=12.04

      Setup:
      Install couchbase server 2.5.0-979 on 12 servers
      Create 12 nodes cluster with zone setup in environment above
      Create 2 buckets, default and sasl with 3 replica at each bucket.
      Load 100 K items to each bucket
      Verified replica distributed correctly

      Tests:
      Shutdown couchbase server in all nodes of group 4
      Failover all down nodes in group 4
      Rebalance. Rebalance completed
      Verified replica distributed correctly
      Verified no data lost.

      Start couchbase server in all nodes of group 4 that I shut them down before.
      Add all 3 nodes to group 4.
      Rebalance ==> Failed (may due to time zone at one added node does not sync. Fixed this time zone)
      Verified time is sync at all nodes
      Rebalance again and again ==> still failed with error

      Rebalance exited with reason {badarg,
      [{erlang,'++',
      [{'EXIT',

      Link to manifest file of this build http://builds.hq.northscale.net/latestbuilds/couchbase-server-enterprise_x86_64_2.5.0-979-rel.deb.manifest.xml

      Attachments

        1. 10.3.3.58-1262013-343-diag.zip
          2.41 MB
        2. 10.3.3.61-1262013-312-diag.zip
          11.94 MB
        3. 10.3.3.63-1262013-322-diag.zip
          11.99 MB
        4. 10.3.3.64-1262013-344-diag.zip
          9.30 MB
        5. 10.3.3.67-1262013-315-diag.zip
          11.81 MB
        6. 10.3.3.71-1262013-336-diag.zip
          12.21 MB
        7. 10.3.3.74-1262013-319-diag.zip
          11.94 MB
        8. 10.3.3.75-1262013-333-diag.zip
          12.13 MB
        9. 10.3.4.219-1262013-326-diag.zip
          12.13 MB
        10. 10.3.4.220-1262013-329-diag.zip
          12.11 MB
        11. 172.23.105.228-1262013-348-diag.zip
          9.20 MB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              anil Anil Kumar (Inactive)
              thuan Thuan Nguyen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty