Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42458

Memcached crash during shutdown of phosphor

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • None
    • Cheshire-Cat
    • memcached
    • None
    • Triaged
    • 1
    • Yes

    Description

      so we currently crash during shutdown:

      2020-11-03T09:10:55.788653+01:00 INFO Deinitialising tracing
      2020-11-03T09:10:55.877420+01:00 CRITICAL Breakpad caught a crash (Couchbase version 7.0.0-0000). Writing crash dump to /home/trond/compile/cheshire-cat/install/var/lib/couchbase/crash/95a2becc-33b9-42b6-ad71349d-99d539eb.dmp before terminating.
      2020-11-03T09:10:55.877441+01:00 CRITICAL Stack backtrace of crashed thread:
      2020-11-03T09:10:55.877527+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached() [0x561d8fe8a000+0x1f9d17]
      2020-11-03T09:10:55.877540+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x40a) [0x561d8fe8a000+0x209b5a]
      2020-11-03T09:10:55.877547+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xd8) [0x561d8fe8a000+0x209eb8]
      2020-11-03T09:10:55.877554+01:00 CRITICAL     /lib/x86_64-linux-gnu/libpthread.so.0() [0x7f73254b5000+0x153c0]
      2020-11-03T09:10:55.877557+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/lib/libphosphor.so(_ZN8phosphor9ChunkLock10lockMasterEv+0xd) [0x7f7325c34000+0x1063d]
      2020-11-03T09:10:55.877560+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/lib/libphosphor.so(_ZN8phosphor8TraceLog12evictThreadsERSt10lock_guardIS0_E+0x34) [0x7f7325c34000+0x170f4]
      2020-11-03T09:10:55.877562+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/lib/libphosphor.so(_ZN8phosphor8TraceLog4stopERSt10lock_guardIS0_Eb+0x39) [0x7f7325c34000+0x17159]
      2020-11-03T09:10:55.877564+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/lib/libphosphor.so(_ZN8phosphor8TraceLog4stopEb+0x4f) [0x7f7325c34000+0x171ff]
      2020-11-03T09:10:55.877572+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached() [0x561d8fe8a000+0xe6dcf]
      2020-11-03T09:10:55.877578+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached() [0x561d8fe8a000+0x94ab2]
      2020-11-03T09:10:55.877595+01:00 CRITICAL     /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f73252c1000+0x270b3]
      2020-11-03T09:10:55.877601+01:00 CRITICAL     /home/trond/compile/cheshire-cat/install/bin/memcached(_start+0x2e) [0x561d8fe8a000+0x8abbe]
      2020-11-03T09:10:55.878485+01:00 INFO ---------- Closing logfile
      

      Looks like it was introduced by http://review.couchbase.org/c/kv_engine/+/139369 (if I revert that patch the problem goes away).

      I recall that there are some "problems" related to threads and Phosphor

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              james.harrison James Harrison (Inactive)
              trond Trond Norbye
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty