Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
4.5.0
-
300M Inital index build time test.
-
Untriaged
-
Unknown
Description
Still collecting data.
Test | 4.0.0-4051 | 4.1.0-5005 | 4.1.1-5914 | 4.5.0-2151 |
---|---|---|---|---|
Initial Index Build time 300M (min) | 488 | 372 | DNF | 633 |
Attachments
Issue Links
- duplicates
-
MB-21542 Initial and incremental indexing in Watson is slower than in Sherlock (FDB)
-
- Closed
-
- relates to
-
MB-19173 Initial and incremental build time regression for 5 indexes (fdb)
-
- Closed
-
-
MB-19629 50M Incremental Index build time is (much) slower in Watson than sherlock
-
- Closed
-
-
MB-16046 Regression in 2i initial index build time with 300M items.
-
- Closed
-
-
MB-19600 Improve the index build time with append-only writes during initial index build even when circular-writes is the write mode for the index
-
- Closed
-
Updates:
I just synced up with Cihan on this and Cihan has synced up with Chiyoung/John on this.
Summary of the discussion:
This is a tradeoff between disk space and index build time for large index files. it only happens with 5 indexes or 300M items. Considering Watson timeframe, this becomes too late to get fixed. We also have a workload to let CB admin to avoid this issue, which is to do append only writes during initial index build time.
Chiyoung is planning on multiple improvement on ForestDB in Spock (multiple writher support) which will improve this.
Based on the above discussion and comments from Cihan in MB19173, I'm moving this bug out of Watson.
Please let us know if you have any comments/concerns.
Thanks,
Qi