Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-22925

Rebalance fails with "Client.Timeout exceeded while awaiting headers"

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 5.0.0
    • 5.0.0
    • secondary-index
    • None
    •  5.0.0-2025
    • Untriaged
    • Unknown

    Description

      1) Create a 3 node cluster : 1 kv, 1 indexer and 1 n1ql

      2) Create multiple secondary indexes(around 50+ on my vagrant VM's). All the indexes are hosted on indexer1 node.

      3) Add one more indexer node , lets say indexer2 and start rebalance. Indexes won't be redistributed to indexer2(as per design) after rebalance completes.

      4) Remove indexer1 and start rebalance again.

      Run queries, create indexes, build indexes and drop indexes in parallel.

      Rebalance failed with

      {service_rebalance_failed,index, {rebalance_failed, {service_error, <<"Post http://127.0.0.1:9102/dropIndex: net/http: request canceled (Client.Timeout exceeded while awaiting headers)">>}}}

      Another important thing to note is that rebalance issues don't get fixed by running "curl  -u Administrator:password http://index_node:9102/cleanupRebalance" on both the indexer nodes.

      Attachments

        1. Screen Shot 2017-02-21 at 3.26.22 PM.png
          647 kB
          Balakumaran Gopal

        Activity

          People

            Balakumaran.Gopal Balakumaran Gopal
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty