Details
-
Bug
-
Resolution: Fixed
-
Critical
-
6.5.0
-
Untriaged
-
Unknown
-
FTS Sprint Dec-07-2018
Description
In scorch, FTS is always storing the docValues for the fields indexed. There is no configurability around this feature. And this could result in index size amplification on disk, especially when the customer isn't interested or using the docValues. ie sort on fields or facet type of queries.
Hence, FTS should provide an option to disable "docValue" option along with the other field options like term vectors, store field etc. The default value for this could still be a checkedBox/enabled, so the user can untick the default option/(disable docValues) to further reduce the index size.
High index size from customer cases have been reported: https://forums.couchbase.com/t/fts-scorch-how-to-compact-compress-scorch-index-files-too-big-files/19037/12
Note the index size comparison between solr/lucene to that FTS.
Configurable docValues looks like the next low hanging fruit index size reduction.