Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39592

[N1QL+FTS] Mandate keyword analyzer for queries that don't use an analyzer

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      Unknown

      Description

      The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:

      • TermQuery
      • PhraseQuery
      • MultiPhraseQuery
      • FuzzyQuery
      • PrefixQuery
      • RegexpQuery
      • WildcardQuery

      This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
       
      This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
       

      Note: this issue in validation (with a non-covered query) could still happen over certain data with must_not (NEG) over Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does NOT specify the index name within options or the analyzer within the query.

      See: https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/

      Here's an example:

      Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"} 
      Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

       

      • Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
      • However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

      The workarounds here are these queries ..

      SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});

      SELECT *
      FROM keyspace
      WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});

       

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            abhinav Abhinav Dangeti created issue -
            abhinav Abhinav Dangeti made changes -
            Field Original Value New Value
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results for a covering and non-covering query without any index options in the SEARCH(..) function.
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results for a covering and non-covering query without any index options in the SEARCH(..) function.
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
             
            This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
             
            till Till Westmann made changes -
            Link This issue blocks MB-38724 [ MB-38724 ]
            till Till Westmann made changes -
            Labels approved-for-6.6.0
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2193 contains n1fty commit ed72668 with commit message:
            MB-39592: Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2193 contains n1fty commit ed72668 with commit message: MB-39592 : Mandating "keyword" analyzer for certain queries
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7710 contains n1fty commit ef994c6 with commit message:
            MB-39592: [BP] Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7710 contains n1fty commit ef994c6 with commit message: MB-39592 : [BP] Mandating "keyword" analyzer for certain queries
            abhinav Abhinav Dangeti made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            abhinav Abhinav Dangeti made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2196 contains n1fty commit ef994c6 with commit message:
            MB-39592: [BP] Mandating "keyword" analyzer for certain queries

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2196 contains n1fty commit ef994c6 with commit message: MB-39592 : [BP] Mandating "keyword" analyzer for certain queries
            abhinav Abhinav Dangeti made changes -
            Link This issue causes MB-39633 [ MB-39633 ]
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.6.0-7733 contains n1fty commit 1ee2f0e with commit message:
            MB-39592: Gather information of default analyzers of dynamic mappings

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7733 contains n1fty commit 1ee2f0e with commit message: MB-39592 : Gather information of default analyzers of dynamic mappings
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-7.0.0-2226 contains n1fty commit 1ee2f0e with commit message:
            MB-39592: Gather information of default analyzers of dynamic mappings

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2226 contains n1fty commit 1ee2f0e with commit message: MB-39592 : Gather information of default analyzers of dynamic mappings
            abhinav Abhinav Dangeti made changes -
            Link This issue relates to MB-39686 [ MB-39686 ]
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
             
            This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            However, this issue could still happen with a Match, MatchPhrase
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            However, this issue could still happen with a Match, MatchPhrase
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query.
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query.
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:field: dept, value: "Pre-sales is inde_
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:field: dept, value: "Pre-sales is inde_
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}
            Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
                       {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}
            Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
                       {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
                 {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
                 {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
               {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword"
            Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");

            Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.

            However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            (1) SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});
            (2) SELECT * from keyspace WHERE SEARCH(keyspace,
               {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}
            Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}
            SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}
            SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
            (2) 
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}
            Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}
            SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}
            SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
            (2) 
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Labels approved-for-6.6.0 approved-for-6.6.0 releasenote
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using "Keyword" analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with +*must_not*+ over Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with +*must_not*+ over Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Description The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            The fields for the following queries need to be set up with "keyword" analyzer within the index for the query to be sargable for the index:
             * TermQuery
             * PhraseQuery
             * MultiPhraseQuery
             * FuzzyQuery
             * PrefixQuery
             * RegexpQuery
             * WildcardQuery

            This change is necessary to assure consistent results during a covering and a non-covering query scenario when no index options are specified within the SEARCH(..) function.
              
             This will also change the behavior for the above mentioned queries, but in a good away - that the user will always see the same result with and without validation.
              

            _Note: this issue in validation (with a non-covered query) could still happen over certain data with +*must_not (NEG)*+ over Match and MatchPhrase queries when the user has an index with a field using non-standard analyzer and does *{color:#de350b}NOT{color}* specify the index name within options or the analyzer within the query._

            _See: [https://forums.couchbase.com/t/n1ql-search-query-result-varies-depending-on-selected-fields/]_

            _Here's an example:_
            {code:java}Index: "field": "dept", "analyzer": "keyword" Data: {"dept": "Pre-sales"}
            Query: SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales");{code}
             
             * Search phase would return the document as the match query (which uses the keyword analyzer) wouldn't match Sales with Pre-sales.
             * However, during the validation phase - since there's no index context, or analyzer explicitly specified for the match query - the default analyzer is used which causes the document to not be returned for the query.

            The workarounds here are these queries ..
            {code:java}SELECT * from keyspace WHERE SEARCH(keyspace, "-dept:Sales", {"index":"index_name"});{code}
            {code:java}SELECT *
            FROM keyspace
            WHERE SEARCH(keyspace, {"must_not": {"disjuncts":[{"field": "dept", "match": "Sales", "analyzer":"keyword"}]}});{code}
             
            abhinav Abhinav Dangeti made changes -
            Labels approved-for-6.6.0 releasenote approved-for-6.6.0 documentation releasenote
            abhinav Abhinav Dangeti made changes -
            Link This issue relates to MB-39887 [ MB-39887 ]
            Hide
            girish.benakappa Girish Benakappa added a comment -

            With this we will have 2 jobs for n1ql/fts/ search funtion/custom-map/
            centos-fts_custom-map-n1ql-rqg-scorch_6.5_P1 -> Index with keyword analyzer and all the query types for text
            centos-fts_custom-map-n1ql-rqg-scorch-match-phrase -> Index with all the analyzers(including custm-analyzer) and only match and match_phrase query types for text

            Show
            girish.benakappa Girish Benakappa added a comment - With this we will have 2 jobs for n1ql/fts/ search funtion/custom-map/ centos-fts_custom-map-n1ql-rqg-scorch_6.5_P1 -> Index with keyword analyzer and all the query types for text centos-fts_custom-map-n1ql-rqg-scorch-match-phrase -> Index with all the analyzers(including custm-analyzer) and only match and match_phrase query types for text
            Hide
            girish.benakappa Girish Benakappa added a comment -

            Closing this issue as these jobs implemented are running fine in 6.6.0-7873 and 7.0.0-2588

            Show
            girish.benakappa Girish Benakappa added a comment - Closing this issue as these jobs implemented are running fine in 6.6.0-7873 and 7.0.0-2588
            girish.benakappa Girish Benakappa made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            abhinav Abhinav Dangeti made changes -
            Link This issue relates to MB-41536 [ MB-41536 ]

              People

              Assignee:
              abhinav Abhinav Dangeti
              Reporter:
              abhinav Abhinav Dangeti
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                    PagerDuty