Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7657

[RN 2.0.1] XDCR docs to replicate for bucket queue is draining unevenly.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0.1
    • Fix Version/s: 3.0
    • Component/s: couchbase-bucket, XDCR
    • Security Level: Public
    • Labels:
    • Environment:
      couchbase-2.0.1-144-rel on Centos 5.4 on both source and destination
    • Flagged:
      Release Note

      Description

      linux cluster
      22-node-ec2-10GB-Linux

      source Cluster ( 15 Nodes)
      create default bucket -3.5 GB per node
      sasl bucket - 2.0 GB per node

      destination Cluster ( 7 Nodes)
      create default bucket - 3.5 GB per node
      sasl bucket - 2.0 GB per node

      **Loading phase
      Source cluster:
      Default bucket
      define a workload that loads 40 M json items , key size 128-512 bytes to default bucket.
      The workload should push the system into a light dgm ( active resident ratio at 90 percent)

      **Access Phase:
      Source cluster:
      Default bucket
      create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 8 hours

      Then start replication:
      set a bi-directional XDCR on default bucket
      Source cluster:
      default bucket
      create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 2 hours

      Destination cluster:
      default bucket
      define a workload that loads 30 M json items(non-conflicting key-sets) , key size 128-512 bytes to default bucket.

      After all the loads on both clusters stop, wait for "XDCR docs to replicate for bucket" drops. But really slow.
      Saw XDCR docs to replicate for bucket queue is draining unevenly on default bucket on source
      The screen shot is from source cluster.

      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-7657=message:MB-7657

        Activity

        Chisheng Chisheng Hong (Inactive) created issue -
        Chisheng Chisheng Hong (Inactive) made changes -
        Field Original Value New Value
        Description linux cluster
        22-node-ec2-10GB-Linux

        source Cluster ( 15 Nodes)
        create default bucket -3.5 GB per node
        sasl bucket - 2.0 GB per node

        destination Cluster ( 7 Nodes)
        create default bucket - 3.5 GB per node
        sasl bucket - 2.0 GB per node

        **Loading phase
        Source cluster:
        Default bucket
        define a workload that loads 40 M json items , key size 128-512 bytes to default bucket.
        The workload should push the system into a light dgm ( active resident ratio at 90 percent)

        **Access Phase:
        Source cluster:
        Default bucket
        create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 8 hours

        Then start replication:
        set a bi-directional XDCR on default bucket
        Source cluster:
        default bucket
        create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 2 hours

        Destination cluster:
        default bucket
        define a workload that loads 30 M json items(non-conflicting key-sets) , key size 128-512 bytes to default bucket.

        After all the loads on both clusters stop, wait for "XDCR docs to replicate for bucket" drops. But really slow.
        Saw XDCR docs to replicate for bucket queue is draining unevenly
        linux cluster
        22-node-ec2-10GB-Linux

        source Cluster ( 15 Nodes)
        create default bucket -3.5 GB per node
        sasl bucket - 2.0 GB per node

        destination Cluster ( 7 Nodes)
        create default bucket - 3.5 GB per node
        sasl bucket - 2.0 GB per node

        **Loading phase
        Source cluster:
        Default bucket
        define a workload that loads 40 M json items , key size 128-512 bytes to default bucket.
        The workload should push the system into a light dgm ( active resident ratio at 90 percent)

        **Access Phase:
        Source cluster:
        Default bucket
        create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 8 hours

        Then start replication:
        set a bi-directional XDCR on default bucket
        Source cluster:
        default bucket
        create:5%, uupdate:10%, get:80%, delete:5% with cache miss:5%, opsPerSec:30000, running for 2 hours

        Destination cluster:
        default bucket
        define a workload that loads 30 M json items(non-conflicting key-sets) , key size 128-512 bytes to default bucket.

        After all the loads on both clusters stop, wait for "XDCR docs to replicate for bucket" drops. But really slow.
        Saw XDCR docs to replicate for bucket queue is draining unevenly on default bucket on source
        The screen shot is from source cluster.
        farshid Farshid Ghods (Inactive) made changes -
        Labels regression
        ketaki Ketaki Gangal made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        ketaki Ketaki Gangal made changes -
        Priority Critical [ 2 ] Major [ 3 ]
        jin Jin Lim (Inactive) made changes -
        Priority Major [ 3 ] Critical [ 2 ]
        ketaki Ketaki Gangal made changes -
        jin Jin Lim (Inactive) made changes -
        Assignee Junyi Xie [ junyi ] Ketaki Gangal [ ketaki ]
        jin Jin Lim (Inactive) made changes -
        Planned End (re-schedule end date based on new assignee)
        jin Jin Lim (Inactive) made changes -
        Summary XDCR docs to replicate for bucket queue is draining unevenly. [RN 2.0.1] XDCR docs to replicate for bucket queue is draining unevenly.
        Flagged Release Note [ 10010 ]
        junyi Junyi Xie (Inactive) made changes -
        Attachment Screen Shot 2013-02-16 at 5.36.01 PM.png [ 16759 ]
        Component/s couchbase-bucket [ 10173 ]
        junyi Junyi Xie (Inactive) made changes -
        Assignee Ketaki Gangal [ ketaki ] Jin Lim [ jin ]
        junyi Junyi Xie (Inactive) made changes -
        Planned End (re-schedule end date based on new assignee)
        junyi Junyi Xie (Inactive) made changes -
        jin Jin Lim (Inactive) made changes -
        Fix Version/s 2.0.2 [ 10418 ]
        jin Jin Lim (Inactive) made changes -
        Planned Start (set to new fixed version's start date)
        Planned End (set to new fixed version's start date)
        jin Jin Lim (Inactive) made changes -
        Priority Critical [ 2 ] Blocker [ 1 ]
        ketaki Ketaki Gangal made changes -
        ketaki Ketaki Gangal made changes -
        farshid Farshid Ghods (Inactive) made changes -
        Assignee Jin Lim [ jin ] Ketaki Gangal [ ketaki ]
        farshid Farshid Ghods (Inactive) made changes -
        Planned End (re-schedule end date based on new assignee)
        tommie Tommie McAfee made changes -
        Attachment 201_kvonly_drain.png [ 16804 ]
        tommie Tommie McAfee made changes -
        Attachment 201_kvonly_drain02.png [ 16805 ]
        jin Jin Lim (Inactive) made changes -
        Fix Version/s 2.0.1 [ 10399 ]
        jin Jin Lim (Inactive) made changes -
        Assignee Ketaki Gangal [ ketaki ] Jin Lim [ jin ]
        jin Jin Lim (Inactive) made changes -
        Planned End (re-schedule end date based on new assignee)
        mikew Mike Wiederhold made changes -
        Sprint Status Current Sprint [ 10027 ] Next Sprint [ 10028 ]
        dipti Dipti Borkar made changes -
        Rank Ranked higher
        mikew Mike Wiederhold made changes -
        Sprint Status Next Sprint [ 10028 ]
        mikew Mike Wiederhold made changes -
        Fix Version/s 2.1 [ 10414 ]
        Fix Version/s 2.0.2 [ 10418 ]
        jin Jin Lim (Inactive) made changes -
        Assignee Jin Lim [ jin ] Ketaki Gangal [ ketaki ]
        ketaki Ketaki Gangal made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Cannot Reproduce [ 5 ]

          People

          • Assignee:
            ketaki Ketaki Gangal
            Reporter:
            Chisheng Chisheng Hong (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes