Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11601

KV only system test. Rebalance hang in swap rebalance due to memory bloating in one node

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Test Blocker
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • centos 6.4 64-bit

    Description

      Install couchbase server 3.0.0-900 to 8 nodes centos 6.4 64-bit.
      Create 7 nodes cluster
      172.23.107.195
      172.23.107.196
      172.23.107.197
      172.23.107.199
      172.23.107.200
      172.23.107.202
      172.23.107.201
      Create 2 buckets: default and sasl-2 with bucket quota 2.5 GB each
      Load 25 M items to each bucket to bring active resident ratio below 80%
      Do access phase in 4 hours with update, delete, exp and set

      Rebalance in node 203 to cluster. Passed
      Rebalance out node 200. Passed
      Swap rebalance. Add node 200 back and remove node 195. Rebalance hang

      Cluster is in hang state now.

      Rebalance hang is caused by memory usage on one node reached or over bucket quota even though total mem_used of cluster is below high water mark.
      Cluster IPs are below and swap rebalance is in hang state now. (add node 200 and remove node 195)

      172.23.107.195
      172.23.107.196
      172.23.107.197
      172.23.107.199
      172.23.107.200
      172.23.107.202
      172.23.107.201

      172.23.107.203

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            thuan Thuan Nguyen
            thuan Thuan Nguyen
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty