Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-22925

Rebalance fails with "Client.Timeout exceeded while awaiting headers"

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 5.0.0
    • 5.0.0
    • secondary-index
    • None
    •  5.0.0-2025
    • Untriaged
    • Unknown

    Description

      1) Create a 3 node cluster : 1 kv, 1 indexer and 1 n1ql

      2) Create multiple secondary indexes(around 50+ on my vagrant VM's). All the indexes are hosted on indexer1 node.

      3) Add one more indexer node , lets say indexer2 and start rebalance. Indexes won't be redistributed to indexer2(as per design) after rebalance completes.

      4) Remove indexer1 and start rebalance again.

      Run queries, create indexes, build indexes and drop indexes in parallel.

      Rebalance failed with

      {service_rebalance_failed,index, {rebalance_failed, {service_error, <<"Post http://127.0.0.1:9102/dropIndex: net/http: request canceled (Client.Timeout exceeded while awaiting headers)">>}}}

      Another important thing to note is that rebalance issues don't get fixed by running "curl  -u Administrator:password http://index_node:9102/cleanupRebalance" on both the indexer nodes.

      Attachments

        1. index_node1.zip
          8.06 MB
        2. index_node2.zip
          17.14 MB
        3. kv.zip
          36.76 MB
        4. n1ql_node.zip
          11.45 MB
        5. Screen Shot 2017-02-21 at 3.26.22 PM.png
          Screen Shot 2017-02-21 at 3.26.22 PM.png
          647 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty