Critical Description
A match query fails to fetch the expected number of results (as seen on ES and UpsideDown/Moss) on a default Scorch index.
Here's the testrunner script, with which this bug was seen:
./testrunner -i INI_FILE.ini get-cbcollect-info=True,get-coredumps=True,stop-on-failure=True,index_type=scorch,GROUP=P0,fts_quota=512 -t fts.stable_topology_fts.StableTopFTS.test_query_type,items=1000,num_queries=10,query_types=match,cluster=D,F,F,compare_es=True,GROUP=P0 |
Here's 1 query that failed in the dev-cluster jenkins job (http://qa.sc.couchbase.com/view/FTS/job/fts-scorch-test-dev/53/console) :
2018-03-15 09:00:51 | INFO | MainProcess | Cluster_Thread | [fts_base.run_fts_query] Running query {"indexName": "default_index", "from": 0, "fields": [], "explain": false, "ctl": {"timeout": 60000, "consistency": {"vectors": {}, "level": ""}}, "query": {"field": "manages.reports", "match": "Solita Hanna Cytheria"}, "size": 10000000} on node: 172.23.105.69: |
2018-03-15 09:00:51 | INFO | MainProcess | Cluster_Thread | [task.execute] Status: {u'successful': 6, u'failed': 0, u'total': 6, u'errors': {}} |
2018-03-15 09:00:51 | INFO | MainProcess | Cluster_Thread | [task.execute] FTS hits for query: {"field": "manages.reports", "match": "Solita Hanna Cytheria"} is 132 (took 21.210091ms) |
2018-03-15 09:00:51 | INFO | MainProcess | Cluster_Thread | [task.execute] ES hits for query: {"query": {"match": {"manages.reports": "Solita Hanna Cytheria"}}} on es_index is 133 (took 4ms) |
2018-03-15 09:00:51 | ERROR | MainProcess | Cluster_Thread | [task.execute] FAIL: FTS hits: 132, while ES hits: 133 |
2018-03-15 09:00:51 | ERROR | MainProcess | Cluster_Thread | [task.execute] FAIL: Following 1 docs were not returned by FTS, but ES, printing 50: [u'emp10000837'] |
This is the document that wasn’t returned (which should've been):
{
|
"salary": 146344.71, |
"name": "Duvessa Jones", |
"mutated": 0, |
"is_manager": true, |
"dept": "Support", |
"join_date": "1988-01-15T05:58:00", |
"manages": { |
"team_size": 10, |
"reports": [ |
"Araceli Juan José", |
"Solita Reed", |
"Kerry-Ann Hall", |
"Keelia Josué", |
"Amara Carter", |
"Gallia Morgan", |
"Lucretia Wells", |
"Darcei Sr.", |
"Chatha Mitchell", |
"Trista Cooper" |
]
|
},
|
"languages_known": [ |
"Arabic", |
"Thai", |
"Chinese" |
],
|
"emp_id": "10000837", |
"type": "emp", |
"email": "duvessa@mcdiabetes.com" |
}
|
Attached are the server logs from the test above: test_1.zip. These are the FTS nodes on the cluster:
172.23.99.64 |
172.23.105.54 |
172.23.105.69 |
NOTE: Its the case of just the one doc that's missing (only when there're no updates). There have been cases where the number of docs is a large value when updates are involved: MB-28753.