Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60393

[CBM] Segmentation fault while restoring an Azure backup due to missing bucket config

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Yes

    Description

      There is a segmentation fault in one. of the Azure backup tests (see attached logs):

       

      ./testrunner -i /data/workspace/debian-p0-backup_recovery-vset00-00-azure-enhance-1-7.1_P0/testexec.17977.ini -p get-coredumps=False,objstore_provider=azure,multi_threads=True,threads_count=4,objstore_provider_region=westus,sirius_url=http://172.23.120.103:4000
      -t ent_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_sanity,objstore_provider=azure,disable-bucket-config=True,items=1000,GROUP=P0,get-coredumps=False,multi_threads=True,threads_count=4,objstore_provider_region=westus,get-cbcollect-info=True,sirius_url=http://172.23.120.103:4000

       

      The test is doing the following cbbackupmgr calls (assuming an Azure storage container):

      /opt/couchbase/bin//cbbackupmgr config --archive az://container6331eac8/archive-172.23.217.204 --repo backup --obj-access-key-id storageaf6db913 --obj-region westus --obj-secret-access-key {ACCESS_KEY} --obj-staging-dir /tmp/cbbackupmgr-staging --disable-bucket-config
       

      backup --archive az://container6331eac8/archive-172.23.217.204 --repo backup --cluster http://172.23.217.204:8091 --username Administrator  --password password  --obj-access-key-id storageaf6db913 --obj-region westus --obj-secret-access-key {ACCESS_KEY} --obj-staging-dir /tmp/cbbackupmgr-staging --no-progress-bar --threads 4  

      /opt/couchbase/bin//cbbackupmgr info --archive az://container6331eac8/archive-172.23.217.204 --obj-access-key-id storageaf6db913 --obj-region westus --obj-secret-access-key {ACCESS_KEY} --obj-staging-dir /tmp/cbbackupmgr-staging --all --json 

      /opt/couchbase/bin//cbbackupmgr restore --archive az://container6331eac8/archive-172.23.217.204 --repo backup --cluster http://172.23.217.204:8091 --username Administrator --password password --start 2024-01-15T19_06_04.765131584-08_00 --end 2024-01-15T19_06_04.765131584-08_00 --obj-access-key-id storageaf6db913 --obj-region westus --obj-secret-access-key {ACCESS_KEY} --obj-staging-dir /tmp/cbbackupmgr-staging --no-progress-bar --threads 4
      

      Restore stacktrace:

      panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0xb0 pc=0x13920bf]
       
      goroutine 1 [running]:
      github.com/couchbase/backup/archive.(*Source).CheckBucketSupportsHistory(0xc00051c540, {0xc0006106c0, 0x7})
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/source.go:441 +0x7f
      github.com/couchbase/backup/plan/cluster/bucket.(*autoCreateCollections).execute(0xc00051c960, {0x7faae0e06838, 0xc00051c540}, {0x7faae0e068e8, 0xc0006ba000})
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/bucket/autocreate_collections.go:289 +0xc4
      github.com/couchbase/backup/plan/cluster/bucket.(*autoCreateCollections).Execute.func3(0x489698?)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/bucket/autocreate_collections.go:259 +0x27
      github.com/couchbase/tools-common/utils/v2/retry.Retryer[...].do(0x1eeb5e0, 0xc0006183d8, 0xc00090e0d0)
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/tools-common/utils/v2@v2.0.2/retry/retryer.go:67 +0xf3
      github.com/couchbase/tools-common/utils/v2/retry.Retryer[...].DoWithContext(0x1eeb5e0, {0x1ee2290, 0x2b26ec0}, 0x1ee50e8?)
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/tools-common/utils/v2@v2.0.2/retry/retryer.go:47 +0x126
      github.com/couchbase/tools-common/utils/v2/retry.Retryer[...].Do(...)
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/tools-common/utils/v2@v2.0.2/retry/retryer.go:34
      github.com/couchbase/backup/plan/cluster/bucket.(*autoCreateCollections).Execute(0xc00051c960, {0x7faae0e06838, 0xc00051c540}, {0x7faae0e068e8, 0xc0006ba000}, 0xc00090e518)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/bucket/autocreate_collections.go:258 +0x4aa
      github.com/couchbase/backup/plan/cluster/bucket.(*Bucket).Execute(0xc0005259e0, {0x7faae0e06838?, 0xc00051c540}, {0x7faae0e068e8?, 0xc0006ba000}, 0x2a9e680?)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/bucket/bucket.go:156 +0x2a7
      github.com/couchbase/backup/plan/cluster.(*Cluster).Execute.func1(0xf8?)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/cluster.go:96 +0x109
      github.com/couchbase/backup/plan/cluster.(*Cluster).Execute(0xc0006665f0, {0x7faae0e06660?, 0xc00051c540}, {0x7faae0e06728?, 0xc0006ba000}, 0xc00090e518)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/cluster/cluster.go:105 +0xe3
      github.com/couchbase/backup/plan.(*Plan).execute(0xc00001a560, {0x1ef3008?, 0xc00051c540}, {0x1ef71a8?, 0xc0006ba000}, {0x0, 0x0})
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/plan.go:126 +0x132
      github.com/couchbase/backup/plan.(*Plan).Execute(0xc000664240?, {0x1ef3008?, 0xc00051c540?}, {0x1ef71a8?, 0xc0006ba000?}, {0x0?, 0x0?})
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/plan/plan.go:60 +0x47
      github.com/couchbase/backup/transferable.(*transferable).Execute(0x1a23a9b?, {0x0?, 0x0?})
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/transferable/transferable.go:39 +0x45
      main.(*RestoreContext).Run(0xc000234c00)
              backup/cmd/cbbackupmgr/restore.go:455 +0x2442
      github.com/couchbase/cbflag.(*Command).parseFlags(0xc0004cbce0, 0xc00063f1c0, {0xc0000401e0, 0x19, 0x16544a0?})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:283 +0x10de
      github.com/couchbase/cbflag.(*Command).parse(0xc0004cbce0, 0xc00063f1c0, {0xc0000401e0?, 0x19, 0x1a})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:111 +0x12a
      github.com/couchbase/cbflag.(*Command).parseCommands(0xc0004cbd50, 0xc00063f1c0, {0xc0000401d0, 0x1a, 0x1b})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:124 +0x19e
      github.com/couchbase/cbflag.(*Command).parse(0xc0004cbd50, 0xc00063f1c0, {0xc0000401d0?, 0x1a, 0x1b})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:109 +0x13d
      github.com/couchbase/cbflag.(*CLI).Parse(0xc0004ed200, {0xc0000401c0, 0x1b, 0x1c})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/cli.go:70 +0x15c
      main.main()
              backup/cmd/cbbackupmgr/main.go:4116 +0xd677
      

      Info segfault:

      panic: runtime error: invalid memory address or nil pointer dereference
      [signal SIGSEGV: segmentation violation code=0x1 addr=0xc8 pc=0x1388d02]
       
      goroutine 1 [running]:
      github.com/couchbase/backup/archive.(*InfoVisitor).visitBucket(0xc0008a5600?, {0xc0008306f0, 0x7}, 0xc0008a56c0)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/info.go:259 +0x142
      github.com/couchbase/backup/archive.(*InfoVisitor).visitBackup(0x17067c0?, {0xc0006b99b0, 0x23}, 0xc0008a5600)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/info.go:234 +0x8f4
      github.com/couchbase/backup/archive.(*InfoVisitor).visitRepo(0xc000764d70, 0xc00042c6c0, 0x0)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/info.go:166 +0x2e7
      github.com/couchbase/backup/archive.(*InfoVisitor).visitArchive(0xc000764d70, 0xc000050020)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/info.go:121 +0x33e
      github.com/couchbase/backup/archive.(*InfoVisitor).Visit(0xc000050020?, 0x0?)
              /home/couchbase/jenkins/workspace/couchbase-server-unix/backup/archive/info.go:50 +0x485
      main.(*InfoContext).Run(0xc000556640)
              backup/cmd/cbbackupmgr/info.go:196 +0xd28
      github.com/couchbase/cbflag.(*Command).parseFlags(0xc000561ea0, 0xc0004ff1c0, {0xc000130020, 0xc, 0x16544a0?})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:283 +0x10de
      github.com/couchbase/cbflag.(*Command).parse(0xc000561ea0, 0xc0004ff1c0, {0xc000130020?, 0xc, 0xc})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:111 +0x12a
      github.com/couchbase/cbflag.(*Command).parseCommands(0xc000578070, 0xc0004ff1c0, {0xc000130010, 0xd, 0xd})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:124 +0x19e
      github.com/couchbase/cbflag.(*Command).parse(0xc000578070, 0xc0004ff1c0, {0xc000130010?, 0xd, 0xd})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/command.go:109 +0x13d
      github.com/couchbase/cbflag.(*CLI).Parse(0xc000577280, {0xc000130000, 0xe, 0xe})
              /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/couchbase/cbflag@v0.0.0-20220325172535-f0f642f857ed/cli.go:70 +0x15c
      main.main()
              backup/cmd/cbbackupmgr/main.go:4116 +0xd677
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              gilad.kalchheim Gilad Kalchheim
              gilad.kalchheim Gilad Kalchheim
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty