Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32547

Swap FTS node rebalance failed (boltdb panic)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 6.5.0
    • 6.0.0
    • fts
    • Untriaged
    • Unknown

    Description

      Build 6.0.0-1693

      Observed that swap rebalance of FTS node in high bucket density test failed with following error-

      Rebalance exited with reason {service_rebalance_failed,fts, {rebalance_failed, {service_error, <<"nodes: sample, res: (*http.Response)(nil), urlUUID: monitor.UrlUUID{Url:\"http://172.23.96.20:8094\", UUID:\"050b9271854fba4a40721562e160e9be\"}, kind: /api/stats, err: Get http://%40fts-cbauth:51f01258286dc0e326539b393b07c7c1@172.23.96.20:8094/api/stats: EOF">>}}}
      

      The setup has 30 buckets and 1 FTS index per bucket and no its queries.

      Tried swap rebalance again on same setup got same error.

      Logs-
      FTS node which is rebalanced out- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.96.20.zip
      FTS node which is rebalanced in- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.15.zip
      Analytics node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.96.23.zip
      Eventing node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.177.zip
      Index+query node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.19.zip
      Index+query node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.20.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.12.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.13.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.14.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            Sreekanth Sivasankaran Sreekanth Sivasankaran (Inactive)
            mahesh.mandhare Mahesh Mandhare (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty