Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-10771

Delta recovery is slower than full recovery. Meanwhile several performance issues encountered.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 3.0
    • 3.0
    • couchbase-bucket, ns_server
    • Security Level: Public
    • Platform = Physical
      OS = CentOS 6.5
      CPU = Intel Xeon E5-2630
      Memory = 64 GB
      Disk = HDD

    Description

      1 of 4 nodes is being re-added after failover.
      100M x 2KB items, 10K mixed ops/sec.

      Steps:
      1. Failover one of nodes.
      2. Add it back.
      3. Enabled delta recovery.
      4. Sleep 20 minutes.
      5. Rebalance cluster.

      Rebalance in "full recovery" mode takes 19 min.
      Rebalance in "delta recovery" mode takes 25 min.

      In the beginning of rebalance ton of Temp OOM errors are thrown.
      Working set is screwed (cache miss rate goes up to 50%) during rebalance.
      Also there is a spike in ops latency due to rapid vbmap changes.

      Attachments

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. MB-11363 Rebalance after failover (delta recovery mode) fails at the early beginning

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-11582 tmp_oom errors due to spike in memory usage during delta recovery

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavelpaulau Pavel Paulau (Inactive)
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty