Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19320

300M initial build time is slower in watson than in sherlock/4.1.0

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 4.5.0
    • 5.0.0
    • forestdb
    • 300M Inital index build time test.
    • Untriaged
    • Unknown

    Description

      Still collecting data.

      Test 4.0.0-4051 4.1.0-5005 4.1.1-5914 4.5.0-2151
      Initial Index Build time 300M (min) 488 372 DNF 633

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Updates:

            I just synced up with Cihan on this and Cihan has synced up with Chiyoung/John on this.

            Summary of the discussion:
            This is a tradeoff between disk space and index build time for large index files. it only happens with 5 indexes or 300M items. Considering Watson timeframe, this becomes too late to get fixed. We also have a workload to let CB admin to avoid this issue, which is to do append only writes during initial index build time.
            Chiyoung is planning on multiple improvement on ForestDB in Spock (multiple writher support) which will improve this.

            Based on the above discussion and comments from Cihan in MB19173, I'm moving this bug out of Watson.

            Please let us know if you have any comments/concerns.

            Thanks,
            Qi

            qi Qi Zhu (Inactive) added a comment - Updates: I just synced up with Cihan on this and Cihan has synced up with Chiyoung/John on this. Summary of the discussion: This is a tradeoff between disk space and index build time for large index files. it only happens with 5 indexes or 300M items. Considering Watson timeframe, this becomes too late to get fixed. We also have a workload to let CB admin to avoid this issue, which is to do append only writes during initial index build time. Chiyoung is planning on multiple improvement on ForestDB in Spock (multiple writher support) which will improve this. Based on the above discussion and comments from Cihan in MB19173, I'm moving this bug out of Watson. Please let us know if you have any comments/concerns. Thanks, Qi

            Assign to Chiyoung. Moved it to Spock.

            Thanks,
            Qi

            qi Qi Zhu (Inactive) added a comment - Assign to Chiyoung. Moved it to Spock. Thanks, Qi

            Hi Mahesh,

            Please review these results after finishing your work on 4.5.1 benchmarking.

            pavelpaulau Pavel Paulau (Inactive) added a comment - Hi Mahesh, Please review these results after finishing your work on 4.5.1 benchmarking.

            Thanks, Pavel.

            Mahesh Mandharehere are 2 related tickets/tests:
            https://issues.couchbase.com/browse/MB-19173
            https://issues.couchbase.com/browse/MB-19629

            It will be great to see the comparison before/after.

            Thanks,
            Qi

            qi Qi Zhu (Inactive) added a comment - Thanks, Pavel. Mahesh Mandhare here are 2 related tickets/tests: https://issues.couchbase.com/browse/MB-19173 https://issues.couchbase.com/browse/MB-19629 It will be great to see the comparison before/after. Thanks, Qi

            Had offline discussion with Pavel and will review these results after FDB changes.

            mahesh.mandhare Mahesh Mandhare (Inactive) added a comment - Had offline discussion with Pavel and will review these results after FDB changes.

            People

              mahesh.mandhare Mahesh Mandhare (Inactive)
              wayne Wayne Siu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty