Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11715

Rebalance after failover with views is slow (>6h), couch_view_group_cleanup crashes

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Blocker
    • 3.0
    • 3.0
    • view-engine
    • Security Level: Public
    • 3.0.0-943

      Platform = Physical
      OS = CentOS 6.5
      CPU = Intel Xeon E5-2630 (24 vCPU)
      Memory = 64 GB
      Disk = 2 x SSD

    Description

      Rebalance after failover, 3 -> 4, 1 bucket x 100M x 2KB, 1 view, 10K mixed ops/sec, 400 queries/sec

      Steps:
      1. "Failover" one node.
      2. Add it back.
      3. Wait 20 minutes
      4. Trigger cluster rebalance, wait for rebalance to finish.

      Rebalance in 2x slower than in beta build, indexing of individual vbuckets takes up to 3.5h:

      http://cbmonitor.sc.couchbase.com/reports/movements/?filename=e1175a17dae049d38215d5b25ed46f72

      couch_view_group_cleanup crashed 3 times during rebalance: "Segmentation fault".

      gdb --ex 't a a bt full' /opt/couchbase/bin/couch_view_group_cleanup /tmp/core.couch_view_grou.113377.leto-s304.1405132791 < /dev/null > gdb.log

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            pavelpaulau Pavel Paulau (Inactive)
            pavelpaulau Pavel Paulau (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty