Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60737

FTS results don't match with SEARCH() function

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Critical
    • 7.6.0
    • 7.6.0
    • fts
    • None
    • 7.6.0-2090

    Description

      Test to reproduce:

      ./testrunner -i /data/workspace/debian-p0-fts-vset00-00-custom-map-n1ql-rqg-scorch-collections-4_7.0_7.6_P1/testexec.126066.ini -p get-cbcollect-info=False,disable_HTP=True,get-logs=False,stop-on-failure=False,cluster=D+F,index_type=scorch,fts_quota=1000,run_via_n1ql=True,custom_map_add_non_indexed_fields=False,EXCLUDE_GROUP=SKIP_FOR_N1QL,text_analyzer=keyword,java_sdk_client=True,GROUP=COLLECTIONS,container_type=collection,get-cbcollect-info=True,sirius_url=http://172.23.120.103:4000 -t fts.stable_topology_fts.StableTopFTS.index_query_custom_mapping,items=1000,custom_map=True,num_custom_analyzers=1,cm_id=0,num_queries=100,compare_es=True,dataset=all,GROUP=BUCKETS;P0;SKIP_FOR_N1QL;N1QL_MATCH_PHRASE;COLLECTIONS 

      Subcomponent:

      custom-map-n1ql-rqg-scorch-collections-4_7.0_7.6_P1 

       

      Running query against fts and es returns 1911 results

      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] ----------------------------------------------------- Query # 12 ----------------------------------------------------
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [fts_base.run_fts_query] Running query {"indexName": "custom_index", "size": 10000000, "from": 0, "explain": false, "query": {"query": "-manages.reports:\"Dominique\""}, "fields": [], "ctl": {"consistency": {"level": "", "vectors": {}}, "timeout": 60000}} on node as 172.23.122.187 : Administrator:
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [on_prem_rest_client.urllib_request] Making a rest request api=http://172.23.122.187:8094/api/index/custom_index/query verb=POST params=b'{"indexName": "custom_index", "size": 10000000, "from": 0, "explain": false, "query": {"query": "-manages.reports:\\"Dominique\\""}, "fields": [], "ctl": {"consistency": {"level": "", "vectors": {}}, "timeout": 60000}}' client_cert=None verify=False
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] Status: {'total': 1, 'failed': 0, 'successful': 1}
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] FTS hits for query: {"query": "-manages.reports:\"Dominique\""} is 1911 (took 1.66682ms)
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [es_base.search] ES query '{'query': {'query_string': {'query': '-manages.reports:"Dominique"'}}}' 
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] ES hits for query: {"query": {"query_string": {"query": "-manages.reports:\"Dominique\""}}} on es_index is 1911 (took 24ms)

      Running same query via SEARCH returns 1000 less results only 911

      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] SUCCESS: Docs returned by FTS = docs returned by ES, doc_ids verified
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] Running N1QL query: select meta().id from default:default.scope1.collection1 where type='emp' and search(default, {"query": "-manages.reports:\"Dominique\""},{"index": "custom_index"})
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [on_prem_rest_client.query_tool] query params : statement=select+meta%28%29.id+from+default%3Adefault.scope1.collection1+where+type%3D%27emp%27+and+search%28default%2C+%7B%22query%22%3A+%22-manages.reports%3A%5C%22Dominique%5C%22%22%7D%2C%7B%22index%22%3A+%22custom_index%22%7D%29
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] N1QL hits for query: "select meta().id from default:default.scope1.collection1 where type='emp' and search(default, {\"query\": \"-manages.reports:\\\"Dominique\\\"\"},{\"index\": \"custom_index\"})" is 911 (took 198.755876ms)
      2024-02-01 21:38:15 | ERROR | MainProcess | Cluster_Thread | [task.execute] FAIL: FTS hits: 1911, while N1QL hits: 911
      2024-02-01 21:38:15 | ERROR | MainProcess | Cluster_Thread | [task.execute] FAIL: Following 1000 doc(s) were not returned by N1QL,but FTS, printing 50: ['wiki_988', 'wiki_45', 'wiki_487', 'wiki_751', 'wiki_373', 'wiki_970', 'wiki_550', 'wiki_647', 'wiki_243', 'wiki_889', 'wiki_478', 'wiki_764', 'wiki_169', 'wiki_762', 'wiki_601', 'wiki_248', 'wiki_28', 'wiki_705', 'wiki_947', 'wiki_309', 'wiki_741', 'wiki_363', 'wiki_986', 'wiki_834', 'wiki_295', 'wiki_419', 'wiki_643', 'wiki_816', 'wiki_848', 'wiki_998', 'wiki_197', 'wiki_382', 'wiki_216', 'wiki_909', 'wiki_927', 'wiki_152', 'wiki_830', 'wiki_338', 'wiki_825', 'wiki_1000', 'wiki_68', 'wiki_211', 'wiki_72', 'wiki_161', 'wiki_733', 'wiki_704', 'wiki_2', 'wiki_27', 'wiki_374', 'wiki_331']
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] Running N1QL query: select meta().id,* from default:default.scope1.collection1 where type='emp' and search(default, {"query": "-manages.reports:\"Dominique\""},{"index": "custom_index"})
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [on_prem_rest_client.query_tool] query params : statement=select+meta%28%29.id%2C%2A+from+default%3Adefault.scope1.collection1+where+type%3D%27emp%27+and+search%28default%2C+%7B%22query%22%3A+%22-manages.reports%3A%5C%22Dominique%5C%22%22%7D%2C%7B%22index%22%3A+%22custom_index%22%7D%29
      2024-02-01 21:38:15 | INFO | MainProcess | Cluster_Thread | [task.execute] N1QL hits for query: "select meta().id,* from default:default.scope1.collection1 where type='emp' and search(default, {\"query\": \"-manages.reports:\\\"Dominique\\\"\"},{\"index\": \"custom_index\"})" is 911 (took 196.288098ms) 

      This happens randomly over 100 queries -> 2 out of 100 queries failed! - 12,#42

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              sarthak.dua Sarthak Dua
              sarthak.dua Sarthak Dua
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty