Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39838

FlexIndex:Flex Query returns 0 results after upgrade

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      Unknown

      Description

      Build - 5.5.6-4733 --> 6.6.0 build 7781

      • Install 5.5.6-4733
      • Load travel-sample
      • Create default index with default typemapping with keyword analyzer

        {
          "type": "fulltext-index",
          "name": "default_index",
          "uuid": "38eb503bf4b582b7",
          "sourceType": "couchbase",
          "sourceName": "travel-sample",
          "sourceUUID": "58899145280249698e7345a0844a2011",
          "planParams": {
            "maxPartitionsPerPIndex": 171
          },
          "params": {
            "doc_config": {
              "docid_prefix_delim": "",
              "docid_regexp": "",
              "mode": "type_field",
              "type_field": "type"
            },
            "mapping": {
              "analysis": {},
              "default_analyzer": "standard",
              "default_datetime_parser": "dateTimeOptional",
              "default_field": "_all",
              "default_mapping": {
                "default_analyzer": "keyword",
                "dynamic": true,
                "enabled": true
              },
              "default_type": "_default",
              "docvalues_dynamic": true,
              "index_dynamic": true,
              "store_dynamic": false,
              "type_field": "_type"
            },
            "store": {
              "indexType": "scorch",
              "kvStoreName": "mossStore"
            }
          },
          "sourceParams": {}
        }
        

      • FTS query with state:California returns 2261 results

        curl -XPOST -H "Content-Type: application/json" \
        http://172.23.121.65:8094/api/index/default_index/query \
        -d '{
          "explain": true,
          "fields": [
            "*"
          ],
          "highlight": {},
          "query": {
            "query": "state:\"California\""
          }
        }'
         
         
        2261 results (4ms server-side)
        
        

      • Upgrade to 6.6.0 build 7781
      • Above FTS query returns 2261 results but Flex query returns 0 results

        select meta().id from `travel-sample` USE INDEX (USING FTS, USING GSI) where state = 'California'
         
        Explain plan:
        {
            "#operator": "Sequence",
            "~children": [
                {
                    "#operator": "IndexFtsSearch",
                    "covers": [
                        "cover (((`travel-sample`.`state`) = \"California\"))",
                        "cover ((meta(`travel-sample`).`id`))"
                    ],
                    "index": "default_index",
                    "index_id": "38eb503bf4b582b7",
                    "keyspace": "travel-sample",
                    "namespace": "default",
                    "search_info": {
                        "field": "\"\"",
                        "options": "{\"index\": \"default_index\"}",
                        "outname": "out",
                        "query": "{\"query\": {\"field\": \"state\", \"term\": \"California\"}, \"score\": \"none\"}"
                    },
                    "using": "fts"
                },
                {
                    "#operator": "Parallel",
                    "~child": {
                        "#operator": "Sequence",
                        "~children": [
                            {
                                "#operator": "Filter",
                                "condition": "cover (((`travel-sample`.`state`) = \"California\"))"
                            },
                            {
                                "#operator": "InitialProject",
                                "result_terms": [
                                    {
                                        "expr": "cover ((meta(`travel-sample`).`id`))"
                                    }
                                ]
                            },
                            {
                                "#operator": "FinalProject"
                            }
                        ]
                    }
                }
            ]
        }
        

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            abhinav Abhinav Dangeti added a comment -

            Hey Girish Benakappa, nice test case. Unfortunately a duplicate of MB-38957 though.

            You see that's a bug that's affected all FTS indexes prior to 6.6, so an index that comes from before that version will carry that bug. Prior to 6.6, the recommendation is to set the default_analyzer at the top level of the index mapping as opposed to at the index mapping level (in you case: default_mapping.default_analyzer).

            Show
            abhinav Abhinav Dangeti added a comment - Hey Girish Benakappa , nice test case. Unfortunately a duplicate of  MB-38957 though. You see that's a bug that's affected all FTS indexes prior to 6.6, so an index that comes from before that version will carry that bug. Prior to 6.6, the recommendation is to set the default_analyzer at the top level of the index mapping as opposed to at the index mapping level (in you case: default_mapping.default_analyzer).
            Hide
            girish.benakappa Girish Benakappa added a comment - - edited

            Abhinav Dangeti

            =============
            Prior to 6.6, the recommendation is to set the default_analyzer at the top level of the index mapping as opposed to at the index mapping level
            =============

            Is this documented somewhere for customers, that they should use default_analyzer at the top level of the index mapping and not set default_analyzer at index mapping level?

            Now, in this case, after upgrade to 6.6, if i rebuild index with default_analyzer even at index mapping level, then everything is fine.

            Ok, for customers prior to 6.6, we have a bug for setting default_analyzer at index mapping level. And isn't the expectation is just upgrade to 6.6 to have the fix without rebuild of indexes manually?

            Show
            girish.benakappa Girish Benakappa added a comment - - edited Abhinav Dangeti ============= Prior to 6.6, the recommendation is to set the default_analyzer at the top level of the index mapping as opposed to at the index mapping level ============= Is this documented somewhere for customers, that they should use default_analyzer at the top level of the index mapping and not set default_analyzer at index mapping level? Now, in this case, after upgrade to 6.6, if i rebuild index with default_analyzer even at index mapping level, then everything is fine. Ok, for customers prior to 6.6, we have a bug for setting default_analyzer at index mapping level. And isn't the expectation is just upgrade to 6.6 to have the fix without rebuild of indexes manually?
            Hide
            abhinav Abhinav Dangeti added a comment -

            This is not a bug that will simply go away with an upgrade. The index will need to be re-built for the tokens to be re-generated using the correct analyzer. So your expectation is not accurate.

            Is this documented somewhere? .. I suppose not - 'cos we found this bug a month ago. I've tagged "releasenote" label on MB-38957, so this will be noted as things fixed with 6.6 (but not added to docs) - if you have any other suggestions here - you should follow up with the release/docs team.

            Show
            abhinav Abhinav Dangeti added a comment - This is not a bug that will simply go away with an upgrade. The index will need to be re-built for the tokens to be re-generated using the correct analyzer. So your expectation is not accurate. Is this documented somewhere? .. I suppose not - 'cos we found this bug a month ago. I've tagged "releasenote" label on MB-38957 , so this will be noted as things fixed with 6.6 (but not added to docs) - if you have any other suggestions here - you should follow up with the release/docs team.
            Hide
            abhinav Abhinav Dangeti added a comment -

            Also your FTS query in the description is not equivalent to the flex query, try this and you should see the issue with just FTS too ..

            {
              "query": {
                "field": "state",
                "term": "California"
              }
            }

            Show
            abhinav Abhinav Dangeti added a comment - Also your FTS query in the description is not equivalent to the flex query, try this and you should see the issue with just FTS too .. { "query" : { "field" : "state" , "term" : "California" } }

              People

              Assignee:
              girish.benakappa Girish Benakappa
              Reporter:
              girish.benakappa Girish Benakappa
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty