Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-31039

Index disk size is high during initial index build

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Epic Link:
    • Is this a Regression?:
      Yes

      Description

      Build 5.1.2-6022
      Coming from https://issues.couchbase.com/browse/MB-30855?focusedCommentId=289810&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-289810

      Observed that while doing initial index build, index disk size goes high and then settles down after some time.
      For a test of 500M docs, single index build-
      5.1.1-5723 takes maximum of 40GB space, whereas 5.1.2-6022(and vulcan, alice) takes ~100GB space at peak.
      Also observed that higher percentage of fragmentation while initial index build.

      Here is cbmonitor graph- http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=nyx_511-5723_build_secondaryindex_5233&snapshot=nyx_512-6022_build_secondaryindex_8740#ac9466ec456c2a28c48d40e1c4725c3e

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            sundar Sundar Sridharan (Inactive) added a comment -

            One idea is to throttle based on amount of free disk space available.

            Show
            sundar Sundar Sridharan (Inactive) added a comment - One idea is to throttle based on amount of free disk space available.
            Hide
            keshav Keshav Murthy added a comment - - edited

            5.5 has parallel log cleaner feature.  Mahesh Mandhare: can you please verify if this is still issue with 5.5 and greater? Thanks.

            Show
            keshav Keshav Murthy added a comment - - edited 5.5 has parallel log cleaner feature.  Mahesh Mandhare : can you please verify if this is still issue with 5.5 and greater? Thanks.
            Hide
            mahesh.mandhare Mahesh Mandhare (Inactive) added a comment -

            Build 6.5.0-2715

            In latest Mad-Hatter build found that disk used for 500M docs 20% RR test is ~90GB(Observed similar usage in 6.0.0-1693) compare to 5.1.1 it was ~45GB.

            Job- http://perf.jenkins.couchbase.com/job/secondary-plasma-dgm/1922/

            cbmonitor- http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=nyx_650-2715_build_secondaryindex_3209#ac9466ec456c2a28c48d40e1c4725c3e 

            Show
            mahesh.mandhare Mahesh Mandhare (Inactive) added a comment - Build 6.5.0-2715 In latest Mad-Hatter build found that disk used for 500M docs 20% RR test is ~90GB(Observed similar usage in 6.0.0-1693) compare to 5.1.1 it was ~45GB. Job- http://perf.jenkins.couchbase.com/job/secondary-plasma-dgm/1922/ cbmonitor- http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=nyx_650-2715_build_secondaryindex_3209#ac9466ec456c2a28c48d40e1c4725c3e  
            Hide
            jliang John Liang added a comment -
            Show
            jliang John Liang added a comment - In 7.0.0-4226, disk size for 500M initial index build is 30G. http://cbmonitor.sc.couchbase.com/reports/html/?snapshot=nyx_700-4226_build_secondaryindex_935e
            Hide
            raju Raju Suravarjjala added a comment -

            Bulk closing all invalid, duplicate and won't fix bugs

            Show
            raju Raju Suravarjjala added a comment - Bulk closing all invalid, duplicate and won't fix bugs

              People

              Assignee:
              srinath.duvuru Srinath Duvuru
              Reporter:
              mahesh.mandhare Mahesh Mandhare (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty