Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48001

[Enforce-TLS]: Cluster doesn't get balanced even after rebalancing the cluster

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 7.1.0
    • 7.1.0
    • query
    • None
    • Centos 7 64 bit; CB EE 7.1.0-1160

    Description

      Summary:
      Rebalance button doesn't get disabled even after rebalancing the cluster many times as seen in the video. I think it is an indexer issue as I see some error messages related to indexer in error.log

      Steps:
      1. I initially had a 1 node KV cluster (.215). I enabled enforce-tls and disabled it a few times, and finally turned off the n2n encryption
      2. Added .217 node with all services on it. Rebalance finished, but the cluster never got balanced.

      +Observations
      +on .217 error.log

      [ns_server:error,2021-08-17T20:36:59.042-07:00,ns_1@172.23.105.217:service_status_keeper_worker<0.284.6>:rest_utils:get_json:57]Request to (indexer) getIndexStatus with headers [] failed: {ok,
                                                                   {{500,
                                                                     "Internal Server Error"},
                                                                    [{"Content-Length",
                                                                      "122"},
                                                                     {"Date",
                                                                      "Wed, 18 Aug 2021 03:36:59 GMT"},
                                                                     {"Content-Type",
                                                                      "application/json"}],
                                                                    <<"{\"code\":\"error\",\"error\":\"Fail to retrieve cluster-wide metadata from index service\",\"failedNodes\":[\"172.23.105.217:8091\"]}">>}}
      [ns_server:error,2021-08-17T20:36:59.042-07:00,ns_1@172.23.105.217:service_status_keeper-index<0.285.6>:service_status_keeper:handle_cast:103]Service service_index returned incorrect status

      Had seen a similar issue with FTS some time ago
      https://issues.couchbase.com/browse/MB-46424
      But this doesn't seem to be an FTS issue since the workaround mentioned on that bug didn't help with this cluster

      Attachments

        For Gerrit Dashboard: MB-48001
        # Subject Branch Project Status CR V

        Activity

          People

            sumedh.basarkod Sumedh Basarkod (Inactive)
            sumedh.basarkod Sumedh Basarkod (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty