Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51428

[Jepsen] Crash due to failing Expects while processing an Abort

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • Morpheus
    • Cheshire-Cat
    • couchbase-bucket

    Description

      A crash was observed during kv-engine-cheshire-cat-jepsen-nightly-796 on node 172.23.107.228 for build Couchbase Server: 7.0.4-7218-1, full backtrace bt-full.txt and Jenkins log consoleText.txt 

      2022-03-13T17:21:05.314399-07:00 CRITICAL *** Fatal error encountered during exception handling ***
      		 
      2022-03-13T17:21:05.385595-07:00 CRITICAL Caught unhandled std::exception-derived exception. what(): GSL: Precondition failure at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:983
      		 
      2022-03-13T17:21:09.511398-07:00 CRITICAL Breakpad caught a crash (Couchbase version 7.0.4-7218). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/02ab548f-a217-4202-ae2772af-a3a69ae6.dmp before terminating.
      		 
      2022-03-13T17:21:09.511441-07:00 CRITICAL Stack backtrace of crashed thread:
      		 
      2022-03-13T17:21:09.511693-07:00 CRITICAL     #0  /opt/couchbase/bin/memcached() [0x400000+0x67f213]
      		 
      2022-03-13T17:21:09.511760-07:00 CRITICAL     #1  /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x6a03ca]
      		 
      2022-03-13T17:21:09.511808-07:00 CRITICAL     #2  /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x6a0708]
      		 
      2022-03-13T17:21:09.511833-07:00 CRITICAL     #3  /lib/x86_64-linux-gnu/libpthread.so.0() [0x7f7e2ac2e000+0x12980]
      		 
      2022-03-13T17:21:09.511877-07:00 CRITICAL     #4  /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7) [0x7f7e285cc000+0x3ee87]
      		 
      2022-03-13T17:21:09.511914-07:00 CRITICAL     #5  /lib/x86_64-linux-gnu/libc.so.6(abort+0x141) [0x7f7e285cc000+0x407f1]
      		 
      2022-03-13T17:21:09.512014-07:00 CRITICAL     #6  /opt/couchbase/lib/libstdc++.so.6() [0x7f7e28f73000+0xa1efc]
      		 
      2022-03-13T17:21:09.512075-07:00 CRITICAL     #7  /opt/couchbase/bin/memcached() [0x400000+0x68cfef]
      		 
      2022-03-13T17:21:09.512152-07:00 CRITICAL     #8  /opt/couchbase/lib/libstdc++.so.6() [0x7f7e28f73000+0xacf96]
      		 
      2022-03-13T17:21:09.512245-07:00 CRITICAL     #9  /opt/couchbase/lib/libstdc++.so.6() [0x7f7e28f73000+0xad001]
      		 
      2022-03-13T17:21:09.512336-07:00 CRITICAL     #10 /opt/couchbase/lib/libstdc++.so.6() [0x7f7e28f73000+0xad294]
      		 
      2022-03-13T17:21:09.512381-07:00 CRITICAL     #11 /opt/couchbase/bin/memcached() [0x400000+0x8f847]
      		 
      2022-03-13T17:21:09.512499-07:00 CRITICAL     #12 /opt/couchbase/bin/memcached() [0x400000+0x4b0c35]
      		 
      2022-03-13T17:21:09.512562-07:00 CRITICAL     #13 /opt/couchbase/bin/memcached() [0x400000+0x4b216c]
      		 
      2022-03-13T17:21:09.512610-07:00 CRITICAL     #14 /opt/couchbase/bin/memcached() [0x400000+0x4a4772]
      		 
      2022-03-13T17:21:09.512645-07:00 CRITICAL     #15 /opt/couchbase/bin/memcached() [0x400000+0x4a48f7]
      		 
      2022-03-13T17:21:09.512684-07:00 CRITICAL     #16 /opt/couchbase/bin/memcached() [0x400000+0x4ab945]
      		 
      2022-03-13T17:21:09.512728-07:00 CRITICAL     #17 /opt/couchbase/bin/memcached() [0x400000+0x2aaf37]
      		 
      2022-03-13T17:21:09.512819-07:00 CRITICAL     #18 /opt/couchbase/bin/memcached() [0x400000+0x2a6080]
      		 
      2022-03-13T17:21:09.512889-07:00 CRITICAL     #19 /opt/couchbase/bin/memcached() [0x400000+0x71ad70]
      		 
      2022-03-13T17:21:09.512936-07:00 CRITICAL     #20 /opt/couchbase/bin/memcached() [0x400000+0x702b2a]
      		 
      2022-03-13T17:21:09.512980-07:00 CRITICAL     #21 /opt/couchbase/bin/memcached() [0x400000+0x71dd29]
      		 
      2022-03-13T17:21:09.513025-07:00 CRITICAL     #22 /opt/couchbase/bin/memcached() [0x400000+0x2a475d]
      		 
      2022-03-13T17:21:09.513120-07:00 CRITICAL     #23 /opt/couchbase/lib/libstdc++.so.6() [0x7f7e28f73000+0xd62f0]
      		 
      2022-03-13T17:21:09.513133-07:00 CRITICAL     #24 /lib/x86_64-linux-gnu/libpthread.so.0() [0x7f7e2ac2e000+0x76db]
      		 
      2022-03-13T17:21:09.513189-07:00 CRITICAL     #25 /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f7e285cc000+0x12161f]

      Attachments

        1. bt-full.txt
          17 kB
        2. consoleText.txt
          14.38 MB

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-50333 [Jepsen] Crash in memcached due to Expects(prepareSeqno >= checkpointManager->getOpenSnapshotStartSeqno());

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              richard.demellow Richard deMellow
              richard.demellow Richard deMellow
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty