Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48683

[magma, 30TB, 1% DGM]: Pure KV test - There is no movement in the docs for swap-rebalance for almost an hour.

    XMLWordPrintable

Details

    Description

      Steps:

      1. Create a 3 node cluster
      2. Create magma bucket and 50 collections.
      3. Create 100000000 items in each 50 collections
      4. Update 100000000 RandomKey keys to create 50 percent fragmentation in each 50 collections
      5. Create 100000000 items in each 50 collections
      6. Update 100000000 RandomKey keys to create 50 percent fragmentation in each 50 collections
      7. Start CRUD load:

        Read Start: 0
        Read End: 100000000
        Update Start: 0
        Update End: 100000000
        Expiry Start: 0
        Expiry End: 0
        Delete Start: 100000000
        Delete End: 200000000
        Create Start: 200000000
        Create End: 300000000
        Final Start: 200000000
        Final End: 300000000
        

      8. Rebalance IN with Loading of docs
      9. Rebalance OUT with Loading of docs
      10. Rebalance SWAP with Loading of docs. After 28 vBuckets moved there is no more docs movement since an hour:

      QE Test

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/magma_temp_job3.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.Hospital.Murphy.test_rebalance,nodes_init=3,graceful=True,skip_cleanup=True,num_items=100000000,num_buckets=1,bucket_names=GleamBook,doc_size=1536,bucket_type=membase,eviction_policy=fullEviction,iterations=5,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,num_collections=50,maxttl=10,num_indexes=100,pc=10,index_nodes=0,cbas_nodes=0,fts_nodes=0,ops_rate=200000,ramQuota=102400,doc_ops=create:update:delete:read,mutation_perc=100,rebl_ops_rate=50000,key_type=RandomKey -m rest'
      

      Attachments

        1. checkpoint_memory_usage_67.png
          261 kB
          James Harrison
        2. expel_decreases.png
          161 kB
          James Harrison
        3. expel_delete_drain.png
          578 kB
          James Harrison
        4. image-2021-09-30-13-50-46-378.png
          573 kB
          Ritesh Agarwal
        5. mem_freed_by_expel.png
          154 kB
          James Harrison
        6. Screenshot 2021-10-01 at 10.54.14.png
          162 kB
          Ben Huddleston
        7. Screenshot 2021-10-01 at 10.58.27.png
          227 kB
          Ben Huddleston
        8. Screenshot 2021-10-01 at 11.05.26.png
          210 kB
          Ben Huddleston
        9. Screenshot 2021-10-01 at 11.15.12.png
          83 kB
          Ben Huddleston

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ritesh.agarwal Ritesh Agarwal
              ritesh.agarwal Ritesh Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty