Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2385

[FTS] Flex Query against FTS Index returns "No Index Available".

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Not a Bug
    • 2.2.1
    • None
    • operator, testing
    • None
    • 1

    Description

      K8S Cluster: GKE, v1.20

      Steps to Reproduce:

      1. Install 2.2.1 CRD
      2. Install 2.2.1 DAC
      3. Install 2.2.1 Operator
      4. Create a 2 node Couchbase Cluster running server version 7.0.1-6102 with all services enabled.
      5. Create a sample bucket `default` on the Cluster.
      6. Load 10000 docs in the bucket using pillowfight loader.
      7. Create FTS index with name as `test1` and bucket as `default`, In Advanced Settings change value of Default Analyzer  from `Standard` to `keyword`
      8. Run this Flex Query against query context:

        select meta().id from `default` USE INDEX(USING FTS); 

        Query Result Outputs:

        [
          {
            "code": 4000,
            "msg": "No index available on keyspace `default`:`default` that matches your query. Use CREATE PRIMARY INDEX ON `default`:`default` to create a primary index, or check that your expected index is online.",
            "query": "select meta().id from `default` USE INDEX(USING FTS);"
          }
        ]

        cbopinfo and Cluster Logs attached.

      https://cb-engineering.s3.amazonaws.com/Fts/collectinfo-2021-08-31T142309-ns_1%40cb-example-0000.cb-example.default.svc.zip
      https://cb-engineering.s3.amazonaws.com/Fts/collectinfo-2021-08-31T142309-ns_1%40cb-example-0001.cb-example.default.svc.zip
      ns_server.error.log:

       ==============================================================================
      couchbase logs (error.log)
      cbbrowse_logs error.log
      ==============================================================================
      [ns_server:error,2021-08-31T14:18:34.535Z,ns_1@cb-example-0000.cb-example.default.svc:<0.499.0>:menelaus_util:reply_server_error:205]Server error during processing: ["web request failed",
                                       {path,
                                        "/_metakv/V2InternalSettings/InternalSettings"},
                                       {method,'GET'},
                                       {type,error},
                                       {what,function_clause},
                                       {trace,
                                        [{misc,compare_secure,
                                          [undefined,
                                           "2264c032018ae6b06a3e1f28c58fe6d1"],
                                          [{file,"src/misc.erl"},{line,2927}]},
                                         {ns_config_auth,authenticate_special,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,94}]},
                                         {ns_config_auth,authenticate,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,104}]},
                                         {menelaus_auth,authenticate,1,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,300}]},
                                         {menelaus_auth,verify_rest_auth,2,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,409}]},
                                         {menelaus_web,perform_action,2,
                                          [{file,"src/menelaus_web.erl"},
                                           {line,1124}]},
                                         {request_throttler,do_request,3,
                                          [{file,"src/request_throttler.erl"},
                                           {line,58}]},
                                         {menelaus_util,handle_request,2,
                                          [{file,"src/menelaus_util.erl"},
                                           {line,216}]}]}]
      [ns_server:error,2021-08-31T14:18:59.300Z,ns_1@cb-example-0000.cb-example.default.svc:service_status_keeper_worker<0.596.0>:rest_utils:get_json:57]Request to (indexer) getIndexStatus with headers [{"If-None-Match",
                                                         "e6789f9a13a311f7"}] failed: {ok,
                                                                                       {{500,
                                                                                         "Internal Server Error"},
                                                                                        [{"Content-Length",
                                                                                          "146"},
                                                                                         {"Date",
                                                                                          "Tue, 31 Aug 2021 14:18:59 GMT"},
                                                                                         {"Content-Type",
                                                                                          "application/json"}],
                                                                                        <<"{\"code\":\"error\",\"error\":\"Fail to retrieve cluster-wide metadata from index service\",\"failedNodes\":[\"cb-example-0001.cb-example.default.svc:8091\"]}">>}}
      [ns_server:error,2021-08-31T14:18:59.300Z,ns_1@cb-example-0000.cb-example.default.svc:service_status_keeper-index<0.608.0>:service_status_keeper:handle_cast:103]Service service_index returned incorrect status
      

      ns_server.debug.log:

      ========================NOTICE REPORT=========================
      {net_kernel,{'EXIT',<0.764.0>,{recv_challenge_reply_failed,{error,closed}}}}
      [ns_server:error,2021-08-31T14:18:34.535Z,ns_1@cb-example-0000.cb-example.default.svc:<0.499.0>:menelaus_util:reply_server_error:205]Server error during processing: ["web request failed",
                                       {path,
                                        "/_metakv/V2InternalSettings/InternalSettings"},
                                       {method,'GET'},
                                       {type,error},
                                       {what,function_clause},
                                       {trace,
                                        [{misc,compare_secure,
                                          [undefined,
                                           "2264c032018ae6b06a3e1f28c58fe6d1"],
                                          [{file,"src/misc.erl"},{line,2927}]},
                                         {ns_config_auth,authenticate_special,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,94}]},
                                         {ns_config_auth,authenticate,2,
                                          [{file,"src/ns_config_auth.erl"},
                                           {line,104}]},
                                         {menelaus_auth,authenticate,1,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,300}]},
                                         {menelaus_auth,verify_rest_auth,2,
                                          [{file,"src/menelaus_auth.erl"},
                                           {line,409}]},
                                         {menelaus_web,perform_action,2,
                                          [{file,"src/menelaus_web.erl"},
                                           {line,1124}]},
                                         {request_throttler,do_request,3,
                                          [{file,"src/request_throttler.erl"},
                                           {line,58}]},
                                         {menelaus_util,handle_request,2,
                                          [{file,"src/menelaus_util.erl"},
                                           {line,216}]}]}]
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:cb_dist<0.758.0>:cb_dist:info_msg:790]cb_dist: Accepted new connection from <0.762.0> DistCtrl #Port<0.65>: {con,
                                                                             #Ref<0.3455796189.3111124994.78127>,
                                                                             inet_tcp_dist,
                                                                             undefined,
                                                                             undefined}
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:net_kernel<0.761.0>:cb_dist:info_msg:790]cb_dist: Accepting connection from acceptor <0.762.0> using module inet_tcp_dist
      [ns_server:debug,2021-08-31T14:18:34.538Z,ns_1@cb-example-0000.cb-example.default.svc:cb_dist<0.758.0>:cb_dist:info_msg:790]cb_dist: Updated connection: {con,#Ref<0.3455796189.3111124994.78127>,
                                        inet_tcp_dist,<0.766.0>,
                                        #Ref<0.3455796189.3111124995.78485>}
      [ns_server:debug,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:<0.747.0>:dist_manager:bringup:267]Attempted to save node name to disk: ok
      [ns_server:debug,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:<0.747.0>:dist_manager:wait_for_node:274]Waiting for connection to node 'babysitter_of_ns_1@cb.local' to be established
      [error_logger:info,2021-08-31T14:18:34.539Z,ns_1@cb-example-0000.cb-example.default.svc:net_kernel<0.761.0>:ale_error_logger_handler:do_log:101] 

      Even though error says nodes failed over, Screenshots attached to highlight both server nodes are available and server logs can be collected through them.

      Memory Quotas Screenshot attached.

      P.S. : The Cluster was accessed using port-forward to create FTS Index.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          After speaking to FTS-QE folks on Slack: Flex Query cannot be open ended, there needs to be a condition against which the flex query will run. Therefore

           select meta().id from `default` USE INDEX(USING FTS);

          needs to be modified. 

          Modified Flex Query:

          select meta().id from `default` USE INDEX(USING FTS) WHERE click= "to edit"; 

          if doc looks like:

          {
            "click": "to edit",
            "with JSON": "there are no reserved field names"
          } 

          The Query here gives out the expected result.

          prateek.kumar Prateek Kumar added a comment - After speaking to FTS-QE folks on Slack: Flex Query cannot be open ended, there needs to be a condition against which the flex query will run. Therefore select meta().id from ` default ` USE INDEX(USING FTS); needs to be modified.  Modified Flex Query: select meta().id from ` default ` USE INDEX(USING FTS) WHERE click= "to edit" ; if doc looks like: { "click" : "to edit" , "with JSON" : "there are no reserved field names" } The Query here gives out the expected result.

          Index Definition:

          {
            "type": "fulltext-index",
            "name": "test1",
            "uuid": "4dc4f65689971973",
            "sourceType": "gocbcore",
            "sourceName": "default",
            "sourceUUID": "16b009e5879636a9dfb9b5cdf90972a2",
            "planParams": {
              "maxPartitionsPerPIndex": 1024,
              "indexPartitions": 1
            },
            "params": {
              "doc_config": {
                "docid_prefix_delim": "",
                "docid_regexp": "",
                "mode": "type_field",
                "type_field": "type"
              },
              "mapping": {
                "analysis": {},
                "default_analyzer": "keyword",
                "default_datetime_parser": "dateTimeOptional",
                "default_field": "_all",
                "default_mapping": {
                  "dynamic": true,
                  "enabled": true
                },
                "default_type": "_default",
                "docvalues_dynamic": false,
                "index_dynamic": true,
                "store_dynamic": false,
                "type_field": "_type"
              },
              "store": {
                "indexType": "scorch",
                "segmentVersion": 15
              }
            },
            "sourceParams": {}
          } 

          prateek.kumar Prateek Kumar added a comment - Index Definition: { "type" : "fulltext-index" , "name" : "test1" , "uuid" : "4dc4f65689971973" , "sourceType" : "gocbcore" , "sourceName" : "default" , "sourceUUID" : "16b009e5879636a9dfb9b5cdf90972a2" , "planParams" : { "maxPartitionsPerPIndex" : 1024 , "indexPartitions" : 1 }, "params" : { "doc_config" : { "docid_prefix_delim" : "" , "docid_regexp" : "" , "mode" : "type_field" , "type_field" : "type" }, "mapping" : { "analysis" : {}, "default_analyzer" : "keyword" , "default_datetime_parser" : "dateTimeOptional" , "default_field" : "_all" , "default_mapping" : { "dynamic" : true , "enabled" : true }, "default_type" : "_default" , "docvalues_dynamic" : false , "index_dynamic" : true , "store_dynamic" : false , "type_field" : "_type" }, "store" : { "indexType" : "scorch" , "segmentVersion" : 15 } }, "sourceParams" : {} }

          People

            prateek.kumar Prateek Kumar
            prateek.kumar Prateek Kumar
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty