Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-50063

decompress crash seen in Plasma-GSI test

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      goroutine 140 [running]:
      github.com/couchbase/plasma.(*Shard).raisePanic(...)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/shard.go:1050
      github.com/couchbase/plasma.(*LSSCtx).raiseCorrupted(0xc0002d8a00, 0xd58ba0, 0xc001f87610)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/lssctx.go:185 +0x206
      github.com/couchbase/plasma.(*Plasma).fatalPanic(0xc0000d4700, 0xc5e7b4, 0x11)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1523 +0x293
      github.com/couchbase/plasma.(*page).checkMutable(0xc003b88040)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/page.go:941 +0x68
      github.com/couchbase/plasma.(*page).Decompress(0xc003b88040, 0x7f173002a800)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/page.go:2044 +0x2f
      github.com/couchbase/plasma.(*pageWalker).Decompress(0xc0036655f8, 0xc003b88040, 0xc003665650)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/page_walker.go:337 +0x1b2
      github.com/couchbase/plasma.(*Plasma).tryPageSwapin(0xc0000d4700, 0xd8c660, 0xc003b88040, 0xc003665600)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1581 +0x1e3
      github.com/couchbase/plasma.(*Plasma).ReadPage(0xc0000d4700, 0x7f177effd3c0, 0xc8e201, 0xc001fa8240, 0xc000196eb8, 0x7f177effd3c0, 0x7f177effaff0, 0xc003088101)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/pagetable.go:202 +0xf5
      github.com/couchbase/plasma.(*Plasma).tryPageRemoval(0xc0000d4700, 0x7f177effaff0, 0xd8c660, 0xc001f6ddc0, 0xc001fa8240)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1129 +0x103
      github.com/couchbase/plasma.(*Plasma).doPageMerge(0xc0000d4700, 0x7f177effaff0, 0xd8c660, 0xc001f6ddc0, 0xc001fa8240, 0x1)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1395 +0xeb
      github.com/couchbase/plasma.(*Plasma).trySMOs2(0xc0000d4700, 0x7f177effaff0, 0xd8c660, 0xc001f6ddc0, 0xc001fa8240, 0xc001f6dd00, 0x190, 0x19, 0xc8, 0x4, ...)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1378 +0x1430
      github.com/couchbase/plasma.(*Plasma).trySMOs(...)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/plasma.go:1256
      github.com/couchbase/plasma.(*Writer).Lookup(0xc001725280, 0x7f1733c53400, 0x1000, 0x1ffb, 0x0)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/writer.go:158 +0x197
      github.com/couchbase/plasma.(*Writer).LookupKV(0xc001725280, 0xc003b82000, 0x1000, 0x1ffb, 0x1000, 0x1000, 0x6, 0x716, 0x0)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/mvcc.go:490 +0x10c
      main.(*WorkerCtx).doRead(0xc001227a40)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/tools/gsi/worker.go:204 +0x212
      main.(*WorkerCtx).Run(0xc001227a40, 0x0)
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/tools/gsi/worker.go:310 +0x8d2
      created by main.(*InstanceCtx).StartWorkers
      /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/tools/gsi/instance.go:209 +0x81
      + grep -Ei panic /data/jenkins/workspace/centos-2i_plasma_API_tests/cheshire-cat/goproj/src/github.com/couchbase/plasma/tools/gsi/output.txt
      panic: : fatal: Page is immutable
      github.com/couchbase/plasma.(*Shard).raisePanic(...)
      github.com/couchbase/plasma.(*Plasma).fatalPanic(0xc0000d4700, 0xc5e7b4, 0x11)

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Ascending order - Click to sort in descending order
          akhil.mundroy Akhil Mundroy added a comment - - edited

          Panic occurs due to failed assertion during decompress. This occurs when:

          1. During page merge, its parent page is in the middle of merge AND
          2. Both DecompressDuringSwapin and CompressAfterSwapin are set to true

          Only one of DecompressDuringSwapin and CompressAfterSwapin can can be set to true. In the test config both are set to true, leading to the panic and this must be corrected. Also, in applyConfigDefaults, it would be good to check if only one is true and disable DecompressDuringSwapin if both are true as a defensive fix.

          akhil.mundroy Akhil Mundroy added a comment - - edited Panic occurs due to failed assertion during decompress. This occurs when: During page merge, its parent page is in the middle of merge AND Both DecompressDuringSwapin and CompressAfterSwapin are set to true Only one of DecompressDuringSwapin and CompressAfterSwapin can can be set to true. In the test config both are set to true, leading to the panic and this must be corrected. Also, in applyConfigDefaults, it would be good to check if only one is true and disable DecompressDuringSwapin if both are true as a defensive fix.
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.1.0-1914 contains plasma commit cbd9bc2 with commit message:
          MB-50063: Disable DecompressDuringSwapin if CompressAfterSwapin is also true

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1914 contains plasma commit cbd9bc2 with commit message: MB-50063 : Disable DecompressDuringSwapin if CompressAfterSwapin is also true
          akhil.mundroy Akhil Mundroy added a comment -

          Fix needs instrumentation to reliably reproduce. Unit test covers this fix. Dmitriy Kalugin-Balashov, can you close this if the issue has not been seen after build couchbase-server-7.1.0-1914.

          akhil.mundroy Akhil Mundroy added a comment - Fix needs instrumentation to reliably reproduce. Unit test covers this fix. Dmitriy Kalugin-Balashov , can you close this if the issue has not been seen after build couchbase-server-7.1.0-1914.

          People

            Dmitriy.Kalugin-Balashov Dmitriy Kalugin-Balashov
            srinath.duvuru Srinath Duvuru
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty