Details
-
Bug
-
Resolution: Fixed
-
Critical
-
5.0.0
-
Untriaged
-
Yes
Description
Build
5.0.0-3217
Level compaction turned on. Pls note, same test runs fine with full compaction.
Consistently reproducible with
Testcase
./testrunner -i INI_FILE.ini -p skip-cleanup=True,cluster=D+F:F,level_compaction=True -t fts.stable_topology_fts.StableTopFTS.index_query_custom_mapping,items=1000,custom_map=True,cm_id=3,num_queries=100,compare_es=True,update=True,delete=True,GROUP=P0
Steps
1. Load 1k docs from EMP dataset
2. Create a custom index with level compaction on, wait for indexing to complete
3. Do some deletes (300). Check if bucket_count=index_count=es_endex_count=700.
4. Then fire 100 random queries
You will notice some queries incorrectly returning IDs of deleted docs although index count = bucket_count of 700.
A sample query that failed -
-----------------------
|
[2017-07-10 11:29:07,719] - [fts_base:1970] INFO - Running query {"indexName": "custom_index", "from": 0, "fields": [], "explain": false, "ctl": {"timeout": 60000, "consistency": {"vectors": {}, "level": ""}}, "query": {"inclusive_max": false, "field": "manages.team_size", "max": 822, "inclusive_min": false, "min": 7}, "size": 10000000} on node: 172.23.109.49: |
[2017-07-10 11:29:07,740] - [task:1281] INFO - Status: {u'successful': 6, u'failed': 0, u'total': 6} |
[2017-07-10 11:29:07,741] - [task:1305] INFO - FTS hits for query: {"inclusive_max": false, "field": "manages.team_size", "max": 822, "inclusive_min": false, "min": 7} is 242 (took 5.864694ms) |
[2017-07-10 11:29:07,760] - [task:1315] INFO - ES hits for query: {"filtered": {"filter": {"range": {"manages.team_size": {"lt": 822, "gt": 7}}}}} on es_index is 175 (took 4ms) |
[2017-07-10 11:29:07,761] - [task:1320] ERROR - FAIL: FTS hits: 242, while ES hits: 175 |
[2017-07-10 11:29:07,761] - [task:1335] ERROR - FAIL: Following 67 doc(s) were not returned by ES,but FTS, printing 50: [u'emp10000733', u'emp10000899', u'emp10000775', u'emp10000793', u'emp10000790', u'emp10000966', u'emp10000752', u'emp10000804', u'emp10000897', u'emp10000896', u'emp10000994', u'emp10000772', u'emp10000997', u'emp10000976', u'emp10000999', u'emp10000974', u'emp10000819', u'emp10000972', u'emp10000930', u'emp10000719', u'emp10000745', u'emp10000747', u'emp10000953', u'emp10000815', u'emp10000873', u'emp10000830', u'emp10000762', u'emp10000916', u'emp10000723', u'emp10000935', u'emp10000806', u'emp10000728', u'emp10000727', u'emp10000769', u'emp10000969', u'emp10000751', u'emp10000768', u'emp10000709', u'emp10000708', u'emp10000788', u'emp10000821', u'emp10000885', u'emp10000983', u'emp10000982', u'emp10000981', u'emp10000987', u'emp10000746', u'emp10000853', u'emp10000952', u'emp10000963'] |
Pls note, all docs with last 3 digits>700 have been deleted.
Attachments
Issue Links
| For Gerrit Dashboard: MB-25211 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 80597,3 | MB-25211 Make custom-indexing test run with compare_es=False | master | testrunner | Status: MERGED | +2 | +1 |
| 80602,4 | MB-25211: Include deletes during partial compactions | master | moss | Status: MERGED | +2 | +1 |