Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49604

Performance: Rebalance out 4->3 hung

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.0.0 GA, same test passed on 7.1.0-1695

      Test: Its a new test to benchmark the smart batching against 7.0.0

      1. Create 6 node cluster (2 kv , 4 indexes)
      2. Load 50M doc
      3. Create 1k indexes (1 partition, mix replica 0,1,2)
      4. Wait for index build
      5. Rebalance out one of the node, it hung for 3 hours. I need to force stop it

      Run: 

      http://perf.jenkins.couchbase.com/job/hemera-temp/701/consoleFull

      19:33:25 2021-11-16T06:03:25 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:27 2021-11-16T06:03:27 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:29 2021-11-16T06:03:29 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:31 2021-11-16T06:03:31 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:33 2021-11-16T06:03:33 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:35 2021-11-16T06:03:35 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:37 2021-11-16T06:03:37 [INFO] Rebalance progress: 59.46666666666666 %
      19:33:39 2021-11-16T06:03:39 [INFO] Rebalance progress: 59.46666666666666 %
      .
      .
      .
      23:28:52 2021-11-16T09:58:52 [INFO] Rebalance progress: 59.46666666666666 %
      23:28:54 2021-11-16T09:58:54 [INFO] Rebalance progress: 59.46666666666666 %
      23:28:56 2021-11-16T09:58:56 [INFO] Rebalance progress: 59.46666666666666 %
      23:28:58 2021-11-16T09:58:58 [INFO] Rebalance progress: 59.46666666666666 %
      23:29:00 2021-11-16T09:59:00 [INFO] Rebalance progress: 59.46666666666666 %
      23:29:02 2021-11-16T09:59:02 [INFO] Rebalance progress: 59.46666666666666 %

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            vikas.chaudhary Vikas Chaudhary
            vikas.chaudhary Vikas Chaudhary
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty