Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48532

[Magma, 15% DGM, 500GB]: Low end volume test - KV Rebalance-IN is extremely slow on 8 core machines.

    XMLWordPrintable

Details

    Description

      Step 1: Create a 4 node cluster
      Step 2: Create required buckets and 50 collections
      Step 3: Create and upsert 5,000,000 items per collection. Total Items: 250,000,000
      Bucket statistics:

      Bucket Type Storage Backend Replicas Durability TTL Items RAM Quota RAM Used Disk Used ARR
      GleamBookUsers0 couchbase magma 1 none 0 250000000 66.41 GiB 46.00 GiB 491.80 GiB 14.5612268

      Step 4: Rebalance IN with Loading of docs. In 17:30:00 hours, rebalance is at 51%

      When ran iotop on 172.23.105.175, there are a lot of reads(200Mbps to 500Mbps) happening but hardly any writes...

      QE Test

      git fetch "http://review.couchbase.org/TAF" refs/changes/59/161059/13 && git checkout FETCH_HEAD
      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/magma_temp_job1.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.Hospital.Murphy.test_rebalance,nodes_init=4,graceful=True,skip_cleanup=True,num_items=2500000,num_buckets=1,bucket_names=GleamBook,doc_size=1024,bucket_type=membase,eviction_policy=fullEviction,iterations=2,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,num_collections=50,maxttl=10,num_indexes=50,pc=25,index_nodes=2,cbas_nodes=0,fts_nodes=0,ops_rate=80000,ramQuota=17000,doc_ops=create:update:delete:read,rebl_ops_rate=10000,key_type=RandomKey -m rest'
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty