Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-11378

Cluster gets stuck in warmup state after re-creation of empty buckets

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 3.0
    • 3.0
    • couchbase-bucket
    • Security Level: Public
    • 3.0.0-798

      Platform = Physical
      OS = CentOS 6.5
      CPU = Intel Xeon E5-2680 v2 (40 vCPU)
      Memory = 256 GB
      Disk = RAID 10 SSD

    Description

      0. 10 nodes
      1. Create 20 buckets
      2. Wait for cluster to become healthy
      3. Sleep 5 minutes
      4. Remove all buckets
      5. Sleep 2.5 minutes
      6. Create 30 buckets

      At this point all servers become "pending" and attempt to auto-failover all nodes reported:

      Could not auto-failover node ('ns_1@172.23.100.17'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.18'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.19'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.20'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.21'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.22'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.23'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.24'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.25'). There was at least another node down.
      Could not auto-failover node ('ns_1@172.23.100.26'). There was at least another node down.

      memcached reports "Too many connections" and I have trouble finding anything in 600M+ diag logs.

      CBEE 2.5.1 works fine.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. MB-10914 {UPR} ::Control connection to memcached on 'ns_1@IP' disconnected with some other crashes before upr_replicator:init/1, upr_proxy:init/1, replication_manager:init/1

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              dliao David Liao (Inactive)
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty