Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32547

Swap FTS node rebalance failed (boltdb panic)

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 6.0.0
    • 6.5.0
    • fts
    • Untriaged
    • Unknown

    Description

      Build 6.0.0-1693

      Observed that swap rebalance of FTS node in high bucket density test failed with following error-

      Rebalance exited with reason {service_rebalance_failed,fts, {rebalance_failed, {service_error, <<"nodes: sample, res: (*http.Response)(nil), urlUUID: monitor.UrlUUID{Url:\"http://172.23.96.20:8094\", UUID:\"050b9271854fba4a40721562e160e9be\"}, kind: /api/stats, err: Get http://%40fts-cbauth:51f01258286dc0e326539b393b07c7c1@172.23.96.20:8094/api/stats: EOF">>}}}
      

      The setup has 30 buckets and 1 FTS index per bucket and no its queries.

      Tried swap rebalance again on same setup got same error.

      Logs-
      FTS node which is rebalanced out- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.96.20.zip
      FTS node which is rebalanced in- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.15.zip
      Analytics node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.96.23.zip
      Eventing node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.177.zip
      Index+query node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.19.zip
      Index+query node- https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-arke-multi-bucket-250/172.23.97.20.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.12.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.13.zip
      KV node- https://s3.amazonaws.com/bugdb/jira/30_bucket_reb_logs/collectinfo-2019-01-09T041824-ns_1%40172.23.97.14.zip

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          mahesh.mandhare Mahesh Mandhare (Inactive) created issue -
          raju Raju Suravarjjala made changes -
          Field Original Value New Value
          Fix Version/s Mad-Hatter [ 15037 ]
          steve Steve Yen made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          steve Steve Yen made changes -
          Actual End 2019-03-15 11:05 (issue has been resolved)
          mahesh.mandhare Mahesh Mandhare (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          steve Steve Yen made changes -
          Summary Swap FTS node rebalance failed Swap FTS node rebalance failed under high bucket density tests
          steve Steve Yen made changes -
          Assignee Keshav Murthy [ keshav ] Sreekanth Sivasankaran [ sreekanth sivasankaran ]
          steve Steve Yen made changes -
          Summary Swap FTS node rebalance failed under high bucket density tests Swap FTS node rebalance failed (boltdb panic)
          steve Steve Yen made changes -
          Resolution Fixed [ 1 ]
          Status Reopened [ 4 ] Resolved [ 5 ]
          steve Steve Yen made changes -
          Actual End 2019-03-15 11:05 2019-07-08 13:13 (issue has been resolved)
          mahesh.mandhare Mahesh Mandhare (Inactive) made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          mahesh.mandhare Mahesh Mandhare (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          mahesh.mandhare Mahesh Mandhare (Inactive) made changes -
          Resolution Fixed [ 1 ]
          Status Reopened [ 4 ] Closed [ 6 ]

          People

            Sreekanth Sivasankaran Sreekanth Sivasankaran
            mahesh.mandhare Mahesh Mandhare (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty