Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62739

[Analytics][K8S][Upgrade]: Rebalance fails with <<"wait for condition on http://cb-example-0000.cb-example.default.svc:9111/analytics/cluster timed out">>

    XMLWordPrintable

Details

    Description

      Steps to reproduce

      1. Created a 3 node cluster on k8s with operator with all services. 
      2. Created a bucket with 0 docs.
      3. On one pod, memcached was killed in a loop. Failover and rebalance failures occurred as expected.
      4. Stopped the memcached kill loop.
      5. Issued a swap rebalance upgrade 
      6. New pods were created and 2 node upgrade was successfully completed.
      7. During the last node swap rebalance, stopped the rebalance manually. Operator retried the rebalance.The swap rebalance fails at this point with a rebalance failure in a loop

      Rebalance exited with reason {service_rebalance_failed,cbas,
      		 
      {worker_died,
      		 
      {'EXIT',<0.24520.10>,
      		 
      {task_failed,rebalance,
      		 
      {service_error,
      		 
      <<"wait for condition on http://cb-example-0000.cb-example.default.svc:9111/analytics/cluster timed out">>}}}}}.
      		 
      Rebalance Operation Id = 2083c0eb064f503b60b507d57e731de7

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            umang.agrawal Umang
            raghav.sk Raghav S K
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty