Loading...

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 5.0.0
Affects Version/s: 5.0.0
Component/s: fts
Labels:
None

Triage:
Untriaged
Is this a Regression?:
No

Description

Build
5.0.0-2012

Testcase
./testrunner -i FTS_INI_FILE.ini get-cbcollect-info=True,get-coredumps=True,get-logs=False,stop-on-failure=False,cluster=D+F,GROUP=ALL -t fts.stable_topology_fts.StableTopFTS.create_simple_default_index,cluster=D+F,items=100000,bucket_type=ephemeral,update=True,expires=30,memory_only=True,GROUP=P0

Steps:
1. 1 node cluster with D+F
2. Load 100K keys on ephemeral bucket
3. create mem-only-index on the bucket

Indexing does not start

[2017-02-16 13:40:38,128] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:40:44,179] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:40:50,231] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:40:56,300] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:02,354] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:08,422] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:14,481] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:20,558] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:26,617] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:32,695] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:38,769] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:44,839] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:50,894] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:41:56,954] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:03,027] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:09,094] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:15,167] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:21,243] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:27,313] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:33,374] - [fts_base:3357] INFO - Docs in bucket = 100000, docs in FTS index 'default_index_1': 0

[2017-02-16 13:42:39,380] - [fts_base:3379] INFO - FTS indexed 0 docs in 2.02 mins

[2017-02-16 13:42:39,441] - [fts_base:3585] INFO - Docs in index default_index_1=0, bucket docs=100000

FAIL

Note: There's hardly any activity on the server at the time of this indexing. We are well below low watermark.

From fts.log (looks like no mutations were sent by DCP) -

bleveIndexStats":{"index":{"analysis_time":0,"batches":0,"deletes":0,"errors":0,"index_time":0,"kv":{"moss":{"TotOnError":0,"TotCloseBeg":0,"TotCloseMergerDone":0,"TotClosePersisterDone":0,"TotCloseLowerLevelBeg":0,"TotCloseLowerLevelEnd":0,"TotCloseEnd":0,"TotSnapshotBeg":362,"TotSnapshotInternalBeg":364,"TotSnapshotInternalEnd":364,"TotSnapshotEnd":362,"TotNewBatch":2,"TotNewBatchTotalOps":1,"TotNewBatchTotalKeyValBytes":2,"TotExecuteBatchBeg":2,"TotExecuteBatchErr":0,"TotExecuteBatchEmpty":0,"TotExecuteBatchWaitBeg":0,"TotExecuteBatchWaitEnd":0,"TotExecuteBatchAwakeMergerBeg":0,"TotExecuteBatchAwakeMergerEnd":0,"TotExecuteBatchEnd":2,"TotNotifyMergerBeg":0,"TotNotifyMergerEnd":0,"TotMergerEnd":0,"TotMergerLoop":3,"TotMergerLoopRepeat":2,"TotMergerAll":1,"TotMergerInternalBeg":1,"TotMergerInternalErr":0,"TotMergerInternalEnd":1,"TotMergerInternalSkip":1,"TotMergerLowerLevelNotify":0,"TotMergerLowerLevelNotifySkip":0,"TotMergerWaitIncomingBeg":1,"TotMergerWaitIncomingStop":0,"TotMergerWaitIncomingEnd":0,"TotMergerWaitIncomingSkip":2,"TotMergerWaitOutgoingBeg":0,"TotMergerWaitOutgoingStop":0,"TotMergerWaitOutgoingEnd":0,"TotMergerWaitOutgoingSkip":0,"TotPersisterLoop":0,"TotPersisterLoopRepeat":0,"TotPersisterWaitBeg":0,"TotPersisterWaitEnd":0,"TotPersisterEnd":1,"TotPersisterLowerLevelUpdateBeg":0,"TotPersisterLowerLevelUpdateErr":0,"TotPersisterLowerLevelUpdateEnd":0,"CurDirtyOps":2,"CurDirtyBytes":286,"CurDirtySegments":1,"CurDirtyTopOps":0,"CurDirtyTopBytes":0,"CurDirtyTopSegments":0,"CurDirtyMidOps":2,"CurDirtyMidBytes":286,"CurDirtyMidSegments":1,"CurDirtyBaseOps":0,"CurDirtyBaseBytes":0,"CurDirtyBaseSegments":0,"CurCleanOps":0,"CurCleanBytes":0,"CurCleanSegments":0}},"num_plain_text_bytes_indexed":0,"term_searchers_finished":0,"term_searchers_started":0,"updates":0},"search_time":0,"searches":0},"basic":{"DocCount":0},"partitions":{"571":{"seq":0,"seqReceived":0,"uuid":"242235822584616"},"562":{"seq":0,"seqReceived":0,"uuid":"137326633278160"},"555":{"seq":0,"seqReceived":0,"uuid":"266602531442256"},"563":{"seq":0,"seqReceived":0,"uuid":"204854942793441"},"548":{"seq":0,"seqReceived":0,"uuid":"275813117099680"},"550":{"seq":0,"seqReceived":0,"uuid":"92281792485301"},"567":{"seq":0,"seqReceived":0,"uuid":"238117096991746"},"570":{"seq":0,"seqReceived":0,"uuid":"133050105485752"},"573":{"seq":0,"seqReceived":0,"uuid":"278587485916487"},"572":{"seq":0,"seqReceived":0,"uuid":"35516760312630"},"560":{"seq":0,"seqReceived":0,"uuid":"20573095439502"},"549":{"seq":0,"seqReceived":0,"uuid":"109025009745171"},"544":{"seq":0,"seqReceived":0,"uuid":"161446171105501"},"566":{"seq":0,"seqReceived":0,"uuid":"248225114204176"},"575":{"seq":0,"seqReceived":0,"uuid":"255405881514829"},"554":{"seq":0,"seqReceived":0,"uuid":"201998418164552"},"565":{"seq":0,"seqReceived":0,"uuid":"273697746995320"},"545":{"seq":0,"seqReceived":0,"uuid":"266579179407312"},"559":{"seq":0,"seqReceived":0,"uuid":"76699586230841"},"568":{"seq":0,"seqReceived":0,"uuid":"135911603757382"},"564":{"seq":0,"seqReceived":0,"uuid":"21542393550558"},"553":{"seq":0,"seqReceived":0,"uuid":"74640582934842"},"557":{"seq":0,"seqReceived":0,"uuid":"187447420791593"},"574":{"seq":0,"seqReceived":0,"uuid":"123071682993168"},"569":{"seq":0,"seqReceived":0,"uuid":"49908344450152"},"561":{"seq":0,"seqReceived":0,"uuid":"60740021483836"},"558":{"seq":0,"seqReceived":0,"uuid":"176872149262748"},"547":{"seq":0,"seqReceived":0,"uuid":"43742552235491"},"546":{"seq":0,"seqReceived":0,"uuid":"39124866603634"},"551":{"seq":0,"seqReceived":0,"uuid":"169864612637688"},"552":{"seq":0,"seqReceived":0,"uuid":"109479849181799"},"556":{"seq":0,"seqReceived":0,"uuid":"176123034881530"}}},"default_index_1_14fa67597ac0f11c_e64340fa":{"pindexStoreStats":{"TimerBatchStore":{"count":        0,"min":        0,"max":        0,"mean":        0.00,"stddev":        0.00,"percentiles":{"median":        0.00,"75%":        0.00,"95%":        0.00,"99%":        0.00,"99.9%":        0.00},"rates":{"1-min":        0.00,"5-min":        0.00,"15-min":        0.00,"mean":        0.00}},"Errors":[]}

2017-02-16T13:43:57.917-08:00 INFO stats: {

    "default:default_index_1:avg_queries_latency": 0,

    "default:default_index_1:batch_merge_count": 0,

    "default:default_index_1:doc_count": 0,

    "default:default_index_1:iterator_next_count": 0,

    "default:default_index_1:iterator_seek_count": 0,

    "default:default_index_1:num_bytes_live_data": 0,

    "default:default_index_1:num_bytes_used_disk": 0,

    "default:default_index_1:num_mutations_to_index": 100000,

    "default:default_index_1:num_pindexes": 32,

    "default:default_index_1:num_pindexes_actual": 32,

    "default:default_index_1:num_pindexes_target": 32,

    "default:default_index_1:num_recs_to_persist": 64,

    "default:default_index_1:reader_get_count": 0,

    "default:default_index_1:reader_multi_get_count": 0,

    "default:default_index_1:reader_prefix_iterator_count": 0,

    "default:default_index_1:reader_range_iterator_count": 0,

    "default:default_index_1:timer_batch_store_count": 0,

    "default:default_index_1:timer_data_delete_count": 0,

    "default:default_index_1:timer_data_update_count": 0,

    "default:default_index_1:timer_opaque_get_count": 2048,

    "default:default_index_1:timer_opaque_set_count": 1024,

    "default:default_index_1:timer_rollback_count": 0,

    "default:default_index_1:timer_snapshot_start_count": 0,

    "default:default_index_1:total_bytes_indexed": 0,

    "default:default_index_1:total_bytes_query_results": 0,

    "default:default_index_1:total_compactions": 0,

    "default:default_index_1:total_queries": 0,

    "default:default_index_1:total_queries_error": 0,

    "default:default_index_1:total_queries_slow": 0,

    "default:default_index_1:total_queries_timeout": 0,

    "default:default_index_1:total_request_time": 0,

    "default:default_index_1:total_term_searchers": 0,

    "default:default_index_1:writer_execute_batch_count": 0,

    "num_bytes_used_ram": 17578072,

    "pct_cpu_gc": 0.0069794674272294814,

    "total_gc": 5341

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

172.23.108.213-20170216-1342-diag.zip
20.69 MB
17/Feb/17 12:54 PM

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[FTS] ephemeral: No docs indexed to a memory-only-index from ephemeral bucket

Details

Description

Attachments

Attachments

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty