Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-34605

cbbackupmgr - compact speed degradation on ForestDB backups

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      Description

      We're seeing a ~100% increase in compact time between 6.0.1/6.0.2 and the latest MadHatter builds. 
       
      Test

      EE compact time elapsed (seconds), 4 nodes, 1 bucket x 100M x 1KB, DGM, Idle

      Results

      6.5.0-3274: 1,294 secs

      6.0.2-2409: 610 secs

       
      Logs for 6.0.2-2409

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-10042/leto-srv-01.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-10042/leto-srv-02.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-10042/leto-srv-03.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-10042/leto-srv-04.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-10042/tools.zip

       
      Logs for 6.5.0-3274

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-tmp1-52/leto-srv-01.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-tmp1-52/leto-srv-02.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-tmp1-52/leto-srv-03.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-tmp1-52/leto-srv-04.perf.couchbase.com.zip
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-leto-tmp1-52/tools.zip

      Comments

      I would imagine this is to do with us having to open many more ForestDB files due to sharding changes. The logs indicate that throughout compact's runtime it is opening and closing many more ForestDB files than in prior versions. 

      It should also be noted that this increase in compact time is offset by a large reduction in base backup size thanks to improvements in backup.
       

      Attachments

        Activity

          People

            carlos.gonzalez Carlos Gonzalez Betancort (Inactive)
            toby.wilds Toby Wilds (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty