Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45320

[Magma] Backup performance tests failed on 7.0.0-4797

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Test Blocker
    • Resolution: Duplicate
    • Affects Version/s: Cheshire-Cat
    • Fix Version/s: 7.0.0
    • Component/s: tools
    • Triage:
      Triaged
    • Story Points:
      1
    • Is this a Regression?:
      Unknown

      Description

      Probem
      Magma backup run failed on build 7.0.0-4797. The backup process was running for 14 hours until I killed it. http://perf.jenkins.couchbase.com/job/rhea-5node2/952/ 

      07:01:02 2021-03-27T07:01:02 [INFO] Running: ./opt/couchbase/bin/cbbackupmgr backup --archive /workspace/backup --repo default --host [http://172.23.97.26|http://172.23.97.26/] --username Administrator --password password --threads 16 --storage sqlite
       
      21:39:19 Build was aborted
      

      I am able to reproduce this issue on VMs. During my investigation, my runs hit two different issues. I have collected logs and uploaded. 

      • backup was stuck:* Backup progress reached 99% and then stuck. I waited for several minutes and didn't see any progress. The run above should hit this issue. That's why it's running for 14 hours. The log file for this issue is cbbackupmgr-collectinfo-backup-2021-03-29t115703.zip.

      root@ubuntu:/tmp/magma_backup# ./opt/couchbase/bin/cbbackupmgr backup --archive /tmp/backup --repo default --host [http://172.23.105.72|http://172.23.105.72/] --username Administrator --password password --threads 16 --storage sqlite
       
      Backing up to '2021-03-29T11_48_51.331337778-07_00'
       
      Transferring key value data for bucket 'bucket-1' at 123.39KiB/s (about 2s remaining)                                                                                                                                   99655 items / 31.87MiB
       
      [===================================================================================================================================================================================================================================  ] 99.13%
       
      ^Cinterrupt
      

      Steps to reproduce
       
      Expectation
      For backup to be successful

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            Hide
            pvarley Patrick Varley added a comment -

            There is a possibility that is being caused by GOCBC-1073. We are going to bump the gocbcore version, it would be worth knowing if you still hit this issue after that.

            Show
            pvarley Patrick Varley added a comment - There is a possibility that is being caused by GOCBC-1073 . We are going to bump the gocbcore version, it would be worth knowing if you still hit this issue after that.
            Hide
            bo-chun.wang Bo-Chun Wang added a comment -

            I have tried the test with build 4814 on VMs, and it works fine. I will do a full scale run to verify it.

            Show
            bo-chun.wang Bo-Chun Wang added a comment - I have tried the test with build 4814 on VMs, and it works fine. I will do a full scale run to verify it.
            Hide
            bo-chun.wang Bo-Chun Wang added a comment -

            I have a good run with build 4814.

            http://perf.jenkins.couchbase.com/job/rhea-5node1/896/

             

            Show
            bo-chun.wang Bo-Chun Wang added a comment - I have a good run with build 4814. http://perf.jenkins.couchbase.com/job/rhea-5node1/896/  
            Hide
            owend Daniel Owen added a comment -

            Resolving as a duplicate of GOCBC-1073

            Show
            owend Daniel Owen added a comment - Resolving as a duplicate of GOCBC-1073
            Hide
            mihir.kamdar Mihir Kamdar added a comment -

            Bulk closing non-fixed defects. Feel free to reopen if necessary.

            Show
            mihir.kamdar Mihir Kamdar added a comment - Bulk closing non-fixed defects. Feel free to reopen if necessary.

              People

              Assignee:
              bo-chun.wang Bo-Chun Wang
              Reporter:
              bo-chun.wang Bo-Chun Wang
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty