Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39592

[N1QL+FTS] Mandate keyword analyzer for queries that don't use an analyzer

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      Unknown

      Description

      The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:

      • TermQuery
      • PhraseQuery
      • MultiPhraseQuery
      • FuzzyQuery
      • PrefixQuery
      • RegexpQuery
      • WildcardQuery

      This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
       
      This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
       

      Note: this issue in validation (with a non-covered query) could still happen over certain data with must_not (NEG) over Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does NOT specify the index name within options or the analyzer within the query.

      See: https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/

      Here's an example:

      Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"} 
      Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

       

      • Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
      • However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

      The workarounds here are these queries ..

      SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});

      SELECT *
      FROM keyspace
      WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});

       

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2193 contains n1fty commit ed72668 with commit message:
            MB-39592: Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2193 contains n1fty commit ed72668 with commit message: MB-39592 : Mandating "keyword" analyzer for certain queries
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7710 contains n1fty commit ef994c6 with commit message:
            MB-39592: [BP] Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7710 contains n1fty commit ef994c6 with commit message: MB-39592 : [BP] Mandating "keyword" analyzer for certain queries
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2196 contains n1fty commit ef994c6 with commit message:
            MB-39592: [BP] Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2196 contains n1fty commit ef994c6 with commit message: MB-39592 : [BP] Mandating "keyword" analyzer for certain queries
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7733 contains n1fty commit 1ee2f0e with commit message:
            MB-39592: Gather information of default analyzers of dynamic mappings

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7733 contains n1fty commit 1ee2f0e with commit message: MB-39592 : Gather information of default analyzers of dynamic mappings
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2226 contains n1fty commit 1ee2f0e with commit message:
            MB-39592: Gather information of default analyzers of dynamic mappings

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2226 contains n1fty commit 1ee2f0e with commit message: MB-39592 : Gather information of default analyzers of dynamic mappings
            Hide
            girish.benakappa Girish Benakappa added a comment -

            With this we will have 2 jobs for n1ql/fts/ search funtion/custom-map/
            centos-fts_custom-map-n1ql-rqg-scorch_6.5_P1 -> Index with keyword analyzer and all the query types for text
            centos-fts_custom-map-n1ql-rqg-scorch-match-phrase -> Index with all the analyzers(including custm-analyzer) and only match and match_phrase query types for text

            Show
            girish.benakappa Girish Benakappa added a comment - With this we will have 2 jobs for n1ql/fts/ search funtion/custom-map/ centos-fts_custom-map-n1ql-rqg-scorch_6.5_P1 -> Index with keyword analyzer and all the query types for text centos-fts_custom-map-n1ql-rqg-scorch-match-phrase -> Index with all the analyzers(including custm-analyzer) and only match and match_phrase query types for text
            Hide
            girish.benakappa Girish Benakappa added a comment -

            Closing this issue as these jobs implemented are running fine in 6.6.0-7873 and 7.0.0-2588

            Show
            girish.benakappa Girish Benakappa added a comment - Closing this issue as these jobs implemented are running fine in 6.6.0-7873 and 7.0.0-2588

              People

              Assignee:
              abhinav Abhinav Dangeti
              Reporter:
              abhinav Abhinav Dangeti
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                    PagerDuty