Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7657

[RN 2.0.1] XDCR docs to replicate for bucket queue is draining unevenly.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0.1
    • Fix Version/s: 3.0
    • Component/s: couchbase-bucket, XDCR
    • Security Level: Public
    • Labels:
    • Environment:
      couchbase-2.0.1-144-rel on Centos 5.4 on both source and destination
    • Flagged:
      Release Note

      Description

      linux cluster
      22-node-ec2-10GB-Linux

      source Cluster ( 15 Nodes)
      create default bucket -3.5 GB per node
      sasl bucket - 2.0 GB per node

      destination Cluster ( 7 Nodes)
      create default bucket - 3.5 GB per node
      sasl bucket - 2.0 GB per node

      **Loading phase
      Source cluster:
      Default bucket
      define a workload that loads 40 M json items , key size 128-512 bytes to default bucket.
      The workload should push the system into a light dgm ( active resident ratio at 90 percent)

      **Access Phase:
      Source cluster:
      Default bucket
      create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 8 hours

      Then start replication:
      set a bi-directional XDCR on default bucket
      Source cluster:
      default bucket
      create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 2 hours

      Destination cluster:
      default bucket
      define a workload that loads 30 M json items(non-conflicting key-sets) , key size 128-512 bytes to default bucket.

      After all the loads on both clusters stop, wait for "XDCR docs to replicate for bucket" drops. But really slow.
      Saw XDCR docs to replicate for bucket queue is draining unevenly on default bucket on source
      The screen shot is from source cluster.

      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-7657=message:MB-7657

        Activity

        Hide
        kzeller kzeller added a comment -

        You're right "uneven"

        Show
        kzeller kzeller added a comment - You're right "uneven"
        Hide
        maria Maria McDuff (Inactive) added a comment -

        will re-visit once multiple reader/writer is in. this is 2.0.2 must fix.

        Show
        maria Maria McDuff (Inactive) added a comment - will re-visit once multiple reader/writer is in. this is 2.0.2 must fix.
        Hide
        maria Maria McDuff (Inactive) added a comment -

        note to qe (maria): once multiple rdr/writer in, QE to run test.

        Show
        maria Maria McDuff (Inactive) added a comment - note to qe (maria): once multiple rdr/writer in, QE to run test.
        Hide
        mikew Mike Wiederhold added a comment -

        Moving to 2.1 since we need to retest with the multi-reader/writer to see if that solves the problem. We can move this back to 2.0.2 if the ep-engine team has time to look at it.

        Show
        mikew Mike Wiederhold added a comment - Moving to 2.1 since we need to retest with the multi-reader/writer to see if that solves the problem. We can move this back to 2.0.2 if the ep-engine team has time to look at it.
        Hide
        jin Jin Lim added a comment -

        QE will retest this once the large scale test started for 2.0.2 and we will determine if improvements made to 2.0.2 (mrw, io batching, etc) addressed the issue or not. Thanks.

        Show
        jin Jin Lim added a comment - QE will retest this once the large scale test started for 2.0.2 and we will determine if improvements made to 2.0.2 (mrw, io batching, etc) addressed the issue or not. Thanks.

          People

          • Assignee:
            ketaki Ketaki Gangal
            Reporter:
            Chisheng Chisheng Hong (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes