Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59739

Shard rebalance out failing as Timeout: Unable to evict enough memory

    XMLWordPrintable

Details

    Description

      Build: 7.6.0-1816 not seen on 7.6.0-1782

      Run: https://perf.jenkins.couchbase.com/job/aether/3477/console 

      2023-11-18T15:41:43.617-08:00, ns_orchestrator:0:info:message(ns_1@172.23.110.72) - Starting rebalance, KeepNodes = ['ns_1@172.23.110.53','ns_1@172.23.110.54',
                                       'ns_1@172.23.110.72','ns_1@172.23.110.55',
                                       'ns_1@172.23.110.56','ns_1@172.23.110.71'], EjectNodes = ['ns_1@172.23.110.73'], Failed over and being ejected nodes = []; no delta recovery nodes; Operation Id = 50cda8441f67a48eff17eaaab3d516f1
      2023-11-18T15:41:43.827-08:00, ns_rebalancer:0:info:message(ns_1@172.23.110.72) - Started rebalancing bucket bucket-1
      2023-11-18T15:41:44.139-08:00, ns_vbucket_mover:0:info:message(ns_1@172.23.110.72) - Bucket "bucket-1" rebalance appears to be swap rebalance
      2023-11-18T15:43:05.594-08:00, ns_orchestrator:0:critical:message(ns_1@172.23.110.72) - Rebalance exited with reason {service_rebalance_failed,index,
                                    {worker_died,
                                     {'EXIT',<0.16437.8>,
                                      {task_failed,rebalance,
                                       {service_error,
                                        <<"Error creating slice Unable to initialize /data/@2i/bucket-1_index45-7_17293896208214475570_0.index/docIndex, err = Timeout: Unable to evict enough memory 37991993232">>}}}}}.
      Rebalance Operation Id = 50cda8441f67a48eff17eaaab3d516f1
       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            devansh.srivastava Devansh Srivastava
            vikas.chaudhary Vikas Chaudhary
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty