Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2385

[FTS] Flex Query against FTS Index returns "No Index Available".

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not a Bug
    • 2.2.1
    • None
    • operator, testing
    • None
    • 1

    Description

      K8S Cluster: GKE, v1.20

      Steps to Reproduce:

      1. Install 2.2.1 CRD
      2. Install 2.2.1 DAC
      3. Install 2.2.1 Operator
      4. Create a 2 node Couchbase Cluster running server version 7.0.1-6102 with all services enabled.
      5. Create a sample bucket `default` on the Cluster.
      6. Load 10000 docs in the bucket using pillowfight loader.
      7. Create FTS index with name as `test1` and bucket as `default`, In Advanced Settings change value of Default Analyzer  from `Standard` to `keyword`
      8. Run this Flex Query against query context:

        select meta().id from `default` USE INDEX(USING FTS); 

        Query Result Outputs:

        [
          {
            "code": 4000,
            "msg": "No index available on keyspace `default`:`default` that matches your query. Use CREATE PRIMARY INDEX ON `default`:`default` to create a primary index, or check that your expected index is online.",
            "query": "select meta().id from `default` USE INDEX(USING FTS);"
          }
        ]

        cbopinfo and Cluster Logs attached.

      https://cb-engineering.s3.amazonaws.com/Fts/collectinfo-2021-08-31T142309-ns_1%40cb-example-0000.cb-example.default.svc.zip
      https://cb-engineering.s3.amazonaws.com/Fts/collectinfo-2021-08-31T142309-ns_1%40cb-example-0001.cb-example.default.svc.zip
      ns_server.error.log:

       ==============================================================================
      couchbase logs (error.log)
      cbbrowse_logs error.log
      ==============================================================================
      [ns_server:error,2021-08-31T14:18:34.535Z,ns_1@cb-example-0000.cb-example.default.svc:<0.499.0>:menelaus_util:reply_server_error:205]Server error during processing: ["web request failed",
                                       {path,
                                        "/_metakv/V2InternalSettings/InternalSettings"},
                                       {method,'GET'},
                                       {type,error},
                                       {what,function_clause},
                                       {trace,
                                        [{misc,compare_secure,
                                          [undefined,
                                           "2264c032018ae6b06a3e1f28c58fe6d1"],
                                          [{file,"src/misc.erl"},{line,2927}]},
                                         {ns_config_auth,authenticate_special,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,94}]},
                                         {ns_config_auth,authenticate,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,104}]},
                                         {menelaus_auth,authenticate,1,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,300}]},
                                         {menelaus_auth,verify_rest_auth,2,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,409}]},
                                         {menelaus_web,perform_action,2,
                                          [{file,"src/menelaus_web.erl"},
                                           {line,1124}]},
                                         {request_throttler,do_request,3,
                                          [{file,"src/request_throttler.erl"},
                                           {line,58}]},
                                         {menelaus_util,handle_request,2,
                                          [{file,"src/menelaus_util.erl"},
                                           {line,216}]}]}]
      [ns_server:error,2021-08-31T14:18:59.300Z,ns_1@cb-example-0000.cb-example.default.svc:service_status_keeper_worker<0.596.0>:rest_utils:get_json:57]Request to (indexer) getIndexStatus with headers [{"If-None-Match",
                                                         "e6789f9a13a311f7"}] failed: {ok,
                                                                                       {{500,
                                                                                         "Internal Server Error"},
                                                                                        [{"Content-Length",
                                                                                          "146"},
                                                                                         {"Date",
                                                                                          "Tue, 31 Aug 2021 14:18:59 GMT"},
                                                                                         {"Content-Type",
                                                                                          "application/json"}],
                                                                                        <<"{\"code\":\"error\",\"error\":\"Fail to retrieve cluster-wide metadata from index service\",\"failedNodes\":[\"cb-example-0001.cb-example.default.svc:8091\"]}">>}}
      [ns_server:error,2021-08-31T14:18:59.300Z,ns_1@cb-example-0000.cb-example.default.svc:service_status_keeper-index<0.608.0>:service_status_keeper:handle_cast:103]Service service_index returned incorrect status
      

      ns_server.debug.log:

      ========================NOTICE REPORT=========================
      {net_kernel,{'EXIT',<0.764.0>,{recv_challenge_reply_failed,{error,closed}}}}
      [ns_server:error,2021-08-31T14:18:34.535Z,ns_1@cb-example-0000.cb-example.default.svc:<0.499.0>:menelaus_util:reply_server_error:205]Server error during processing: ["web request failed",
                                       {path,
                                        "/_metakv/V2InternalSettings/InternalSettings"},
                                       {method,'GET'},
                                       {type,error},
                                       {what,function_clause},
                                       {trace,
                                        [{misc,compare_secure,
                                          [undefined,
                                           "2264c032018ae6b06a3e1f28c58fe6d1"],
                                          [{file,"src/misc.erl"},{line,2927}]},
                                         {ns_config_auth,authenticate_special,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,94}]},
                                         {ns_config_auth,authenticate,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,104}]},
                                         {menelaus_auth,authenticate,1,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,300}]},
                                         {menelaus_auth,verify_rest_auth,2,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,409}]},
                                         {menelaus_web,perform_action,2,
                                          [{file,"src/menelaus_web.erl"},
                                           {line,1124}]},
                                         {request_throttler,do_request,3,
                                          [{file,"src/request_throttler.erl"},
                                           {line,58}]},
                                         {menelaus_util,handle_request,2,
                                          [{file,"src/menelaus_util.erl"},
                                           {line,216}]}]}]
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:cb_dist<0.758.0>:cb_dist:info_msg:790]cb_dist: Accepted new connection from <0.762.0> DistCtrl #Port<0.65>: {con,
                                                                             #Ref<0.3455796189.3111124994.78127>,
                                                                             inet_tcp_dist,
                                                                             undefined,
                                                                             undefined}
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:net_kernel<0.761.0>:cb_dist:info_msg:790]cb_dist: Accepting connection from acceptor <0.762.0> using module inet_tcp_dist
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:cb_dist<0.758.0>:cb_dist:info_msg:790]cb_dist: Updated connection: {con,#Ref<0.3455796189.3111124994.78127>,
                                        inet_tcp_dist,<0.766.0>,
                                        #Ref<0.3455796189.3111124995.78485>}
      [ns_server:debug,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:<0.747.0>:dist_manager:bringup:267]Attempted to save node name to disk: ok
      [ns_server:debug,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:<0.747.0>:dist_manager:wait_for_node:274]Waiting for connection to node 'babysitter_of_ns_1@cb.local' to be established
      [error_logger:info,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:net_kernel<0.761.0>:ale_error_logger_handler:do_log:101] 

      Even though error says nodes failed over, Screenshots attached to highlight both server nodes are available and server logs can be collected through them.

      Memory Quotas Screenshot attached.

      P.S. : The Cluster was accessed using port-forward to create FTS Index.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          No work has yet been logged on this issue.

          People

            prateek.kumar Prateek Kumar
            prateek.kumar Prateek Kumar
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty