Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-59098

Test the performance of cbdatarecovery on magma buckets

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • Major
    • 7.6.0
    • master
    • tools
    • None
    • 0

    Description

      The recovery of magma buckets has been enabled by this change: https://issues.couchbase.com/browse/MB-49475.

      We should test the memory and CPU usage when recovering a magma bucket.

      I have already tested this on my local machine with with a magma bucket that contains 1 shard and another one that contains 8 shards. The tests results are attached to this ticket with the script used to gather that data. NOTE: the script `mem_count_local_mac.bash` has been edited to work specifically on MacOS if on linux use `mem_count_linux.bash`

      Further testing needs to be done with clusters that have around 100GB of data on them. The testing steps should be as follows:

      1. Spin up a couchbase server on an AWS instance.
      2. Create a magma bucket and using `cbpillowfight` generate 100GB of data on that bucket.
      3. Run the script for collecting memory and CPU data (attached) and then run `cbdatarecovery`.
      4. Collect the data.

      We should do this for a magma bucket with 1 shard and 8 shards. We should then compare the results with the local run. We should determine whether the amount of data on magma affects the performance of `cbdatarecovery` and if it does look for ways to avoid/improve this.

      Attachments

        1. couchstore_to_couchstore.txt
          8 kB
        2. couchstore_to_magma.txt
          39 kB
        3. local_mem1.tsv
          2 kB
        4. local_mem8.tsv
          1 kB
        5. magma_to_couchstore.txt
          14 kB
        6. magma_to_magma.txt
          38 kB
        7. mem_count_linux.bash
          0.1 kB
        8. mem_count_local_mac.bash
          0.1 kB
        9. set_latency_couchstore_vs_magma.png
          set_latency_couchstore_vs_magma.png
          1.71 MB
        10. set_with_meta_latency_couchstore_vs_magma.png
          set_with_meta_latency_couchstore_vs_magma.png
          167 kB
        11. set_with_meta logs.zip
          13.44 MB
        12. set logs.zip
          11.06 MB

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              safian.ali Safian Ali
              Gvidas.Razevicius Gvidas Razevicius
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty