Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39472

Increase in Initial and Incremental index for no mutations during initial build only ,MOI for 1B documents

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: Cheshire-Cat, 6.6.0
    • Fix Version/s: None
    • Component/s: test-execution
    • Environment:
       6.6.0-7557, 7.0.0-1896
    • Story Points:
      1

      Description

      Tests run :
      1.Initial index (min), 1 bucket x 1B x 1KB, single 2i index, 100K ops/sec, no mutations during initial build only, MOI

      2.Incremental index (min), 1 bucket x 1B x 1KB, single 2i index, 100K ops/sec, no mutations during initial build only, MOI

      Measurements 
      For initial index :
        

      Build Initial index time (min) Link to the job
      6.6.0-7530 23.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6015/
      6.6.0-7557 70.5 http://perf.jenkins.couchbase.com/job/secondary-moi/6105/
      7.0.0-1834 24.2 http://perf.jenkins.couchbase.com/job/secondary-moi/5997/
      7.0.0-1896 77 http://perf.jenkins.couchbase.com/job/secondary-moi/6124/
      7.0.0-1963 73 http://perf.jenkins.couchbase.com/job/secondary-moi/6074/

       

      Changelog between builds 1834 and 1896 :
      http://172.23.123.43:10000/?product=couchbase-server&release=cheshire-cat&version=7.0.0&fromBuild=1834&toBuild=1896&f_asterixdb=off&f_backup=off&f_build=off&f_cbas-ui=off&f_cbbs=off&f_cbft=off&f_cbgt=off&f_eventing=off&f_go_json=off&f_goxdcr=off&f_indexing=on&f_kv_engine=off&f_magma=off&f_ns_server=off&f_platform=off&f_query=off&f_query-ee=off&f_query-ui=off&f_testrunner=off&f_vellum=off

       

      For incremental index :

      Build Incremental index time (min) Link to the job
      6.6.0-7530 163.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6015/
      6.6.0-7557 286.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6105/
      7.0.0-1834 164 http://perf.jenkins.couchbase.com/job/secondary-moi/5997/
      7.0.0-1963 263.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6074/

        Attachments

        1. 172.23.97.5-data-size.png
          172.23.97.5-data-size.png
          89 kB
        2. 172.23.97.5-disk-size.png
          172.23.97.5-disk-size.png
          120 kB
        3. 172.23.97.5-fragmentation.png
          172.23.97.5-fragmentation.png
          96 kB
        4. 172.23.97.7-data-size.png
          172.23.97.7-data-size.png
          88 kB
        5. 172.23.97.7-disk-size.png
          172.23.97.7-disk-size.png
          102 kB
        6. 172.23.97.7-fragmentation.png
          172.23.97.7-fragmentation.png
          90 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          owend Daniel Owen added a comment - - edited

          We can see from the data_size that both .5 and .7 are storing the same amount of data.

          However the disk size for .7 is much bigger. Therefore the fragmentation stats differ greatly.

          Show
          owend Daniel Owen added a comment - - edited We can see from the data_size that both .5 and .7 are storing the same amount of data. However the disk size for .7 is much bigger. Therefore the fragmentation stats differ greatly.
          Hide
          owend Daniel Owen added a comment -

          This lead me to think that it was an issue with compaction not running however compaction has affectively been disabled.

           {database_fragmentation_threshold,{100,undefined}},
          

          So it looks like an issue with the underlying system/hardware.
          One thing that would be good to confirm is that the couchstore files are being deleted prior to running the test, as I see the fragmentation jumping almost immediately up to ~75% on .7. So want to rule out that previous files are not being picked-up.

          Show
          owend Daniel Owen added a comment - This lead me to think that it was an issue with compaction not running however compaction has affectively been disabled. {database_fragmentation_threshold,{100,undefined}}, So it looks like an issue with the underlying system/hardware. One thing that would be good to confirm is that the couchstore files are being deleted prior to running the test, as I see the fragmentation jumping almost immediately up to ~75% on .7. So want to rule out that previous files are not being picked-up.
          Hide
          wayne Wayne Siu added a comment -

          Jyotsna Nayak

          See Dan's comment above.  Can you check?  Thanks.

          Show
          wayne Wayne Siu added a comment - Jyotsna Nayak See Dan's comment above.  Can you check?  Thanks.
          Hide
          jyotsna.nayak Jyotsna Nayak added a comment -

          Sure Wayne, Will look into this

          Show
          jyotsna.nayak Jyotsna Nayak added a comment - Sure Wayne, Will look into this
          Hide
          wayne Wayne Siu added a comment -

          Confirmed that it's an environment issue.  After cleaning up the /data directory, the index build time is back to the level that is expected.

          Build Initial index time (min) Link to the job
          6.6.0-7530 23.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6015/
          6.6.0-7557 70.5 http://perf.jenkins.couchbase.com/job/secondary-moi/6105/
          6.6.0-7557 25.1 http://perf.jenkins.couchbase.com/job/secondary-moi/6318/
          6.6.0-7813 24.4 http://perf.jenkins.couchbase.com/job/secondary-moi/6320/

           Closing the ticket.

           

           

          Show
          wayne Wayne Siu added a comment - Confirmed that it's an environment issue.  After cleaning up the /data directory, the index build time is back to the level that is expected. Build Initial index time (min) Link to the job 6.6.0-7530 23.9 http://perf.jenkins.couchbase.com/job/secondary-moi/6015/ 6.6.0-7557 70.5 http://perf.jenkins.couchbase.com/job/secondary-moi/6105/ 6.6.0-7557 25.1 http://perf.jenkins.couchbase.com/job/secondary-moi/6318/ 6.6.0-7813 24.4 http://perf.jenkins.couchbase.com/job/secondary-moi/6320/  Closing the ticket.    

            People

            Assignee:
            wayne Wayne Siu
            Reporter:
            jyotsna.nayak Jyotsna Nayak
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty