Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4358

sending backoff to tap producer when disk write queue > 1M is not optimal for large cluster with> 100M items and lot of RAM

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • 2.0-beta
    • 1.7.2
    • couchbase-bucket
    • Security Level: Public
    • None

    Description

      right now we stop doing more rebalancing/replicating if the destination node's disk write queue has more than 1 million items

      this number doesnt fit well if user has hundreds of million of items in the cluster .
      for instance if you have a cluster with 120 million items it means each vbucket has 600k items and even moving one bucket at a time will cause the disk write queue to be more than 1M immediately considering ec2 disk performance

      also some users they have deployments with enough RAM that they can keep more items in memory than others.

      so disk write queue limit should be decided based on number of items in the cluster
      amount of RAM and other factors.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            chiyoung Chiyoung Seo (Inactive)
            farshid Farshid Ghods (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty