Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6573

[longevity] rebalance failed due to error "Resetting rebalance status since it's not really running" when there are major page faults on some of the nodes in the cluster

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta
    • Fix Version/s: 2.0
    • Component/s: ns_server
    • Security Level: Public
    • Environment:
      centos 6.2 64bit build 2.0.0-1697

      Description

      Cluster information:

      • 11 centos 6.2 64bit server with 4 cores CPU
      • Each server has 10 GB RAM and 150 GB disk.
      • 8 GB RAM for couchbase server at each node (80% total system memmories)
      • Disk format ext3 on both data and root
      • Each server has its own drive, no disk sharing with other server.
      • Load 7 million items to both buckets
      • Cluster has 2 buckets, default (3GB) and saslbucket (3GB)
      • Each bucket has one doc and 2 views for each doc (default d1 and saslbucket d11)
      • Maintain load about 10 K ops, query view on both doc

      10.3.121.13
      10.3.121.14
      10.3.121.15
      10.3.121.16
      10.3.121.17
      10.3.121.20
      10.3.121.22
      10.3.121.24
      10.3.121.25
      10.3.121.23

      Create cluster with 10 nodes
      Do swap rebalance. Add node 26 and remove node 25.
      Before and during rebalance, cluster does not go into swap.

      Rebalance failed with error "Resetting rebalance status since it's not really running"

      Link to diags of all nodes https://s3.amazonaws.com/packages.couchbase/diag-logs/orange/201209/11ndoes-1697-reb-failed-reset-reb-20120907.tgz

      Link to atop of all nodes https://s3.amazonaws.com/packages.couchbase/atop-files/orange/201209/atop-11nodes-1697-reb-failed-reset-reb-20120907.tgz

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        thuan Thuan Nguyen created issue -
        thuan Thuan Nguyen made changes -
        Field Original Value New Value
        Assignee Chiyoung Seo [ chiyoung ] Aleksey Kondratenko [ alkondratenko ]
        Component/s ns_server [ 10019 ]
        Component/s couchbase-bucket [ 10173 ]
        farshid Farshid Ghods (Inactive) made changes -
        Labels longevity
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Farshid Ghods [ farshid ]
        karan Karan Kumar (Inactive) made changes -
        Assignee Farshid Ghods [ farshid ] Thuan Nguyen [ thuan ]
        farshid Farshid Ghods (Inactive) made changes -
        Assignee Thuan Nguyen [ thuan ] Karan Kumar [ karan ]
        karan Karan Kumar (Inactive) made changes -
        Labels longevity system-test
        farshid Farshid Ghods (Inactive) made changes -
        Summary [longevity] rebalance failed due to error "Resetting rebalance status since it's not really running" [longevity] rebalance failed due to error "Resetting rebalance status since it's not really running" when there are major page faults on some of the nodes in the cluster
        farshid Farshid Ghods (Inactive) made changes -
        Labels system-test 2.0-beta-release-notes system-test
        farshid Farshid Ghods (Inactive) made changes -
        Fix Version/s 2.0 [ 10114 ]
        Fix Version/s 2.0-beta [ 10113 ]
        karan Karan Kumar (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            karan Karan Kumar (Inactive)
            Reporter:
            thuan Thuan Nguyen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes