Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41639

Server API stores storage thread callback to bucket post deletion resulting in a crash if called

    XMLWordPrintable

Details

    • Triaged
    • 1
    • Yes
    • KV-Engine 2021-March

    Description

      172.23.122.111 : Stack Trace of first crash: 18b10b7c-1983-4573-0b8ab49d-804306f3.dmp
      Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'.
       #0  0x00007f74d7dc9d20 in MagmaKVStoreConfig::setStorageThreads(unsigned long) () from /opt/couchbase/bin/../lib/libep.so
       #0  0x00007f74d7dc9d20 in MagmaKVStoreConfig::setStorageThreads(unsigned long) () from /opt/couchbase/bin/../lib/libep.so
       #1  0x0000000000444723 in CreateBucketThread::create() ()
       #2  0x0000000000444c79 in CreateBucketThread::run() ()
       #3  0x00007f74d66445df in Couchbase::Thread::thread_entry() () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0
       #4  0x00007f74d6639947 in platform_thread_wrap(void*) () from /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0
       #5  0x00007f74d3e7ee65 in start_thread () from /lib64/libpthread.so.0
       #6  0x00007f74d3ba788d in clone () from /lib64/libc.so.6
       
      172.23.122.111 : Looking for CRITICAL messages in log
      172.23.122.111 : Found message in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000002.txt
      2020-09-23T16:58:46.487095-07:00 CRITICAL Breakpad caught a crash (Couchbase version 7.0.0-3193). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/18b10b7c-1983-4573-0b8ab49d-804306f3.dmp before terminating.
      2020-09-23T16:58:46.487106-07:00 CRITICAL Stack backtrace of crashed thread:
      2020-09-23T16:58:46.487327-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x19428d]
      2020-09-23T16:58:46.487343-07:00 CRITICAL     /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x1aa60a]
      2020-09-23T16:58:46.487355-07:00 CRITICAL     /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x1aa948]
      2020-09-23T16:58:46.487364-07:00 CRITICAL     /lib64/libpthread.so.0() [0x7f74d3e77000+0xf5f0]
      2020-09-23T16:58:46.487375-07:00 CRITICAL     /opt/couchbase/bin/../lib/libep.so() [0x7f74d7b5f000+0x26ad20]
      2020-09-23T16:58:46.487386-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x44723]
      2020-09-23T16:58:46.487395-07:00 CRITICAL     /opt/couchbase/bin/memcached() [0x400000+0x44c79]
      2020-09-23T16:58:46.487404-07:00 CRITICAL     /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0(_ZN9Couchbase6Thread12thread_entryEv+0xf) [0x7f74d6629000+0x1b5df]
      2020-09-23T16:58:46.487410-07:00 CRITICAL     /opt/couchbase/bin/../lib/libplatform_so.so.0.1.0() [0x7f74d6629000+0x10947]
      2020-09-23T16:58:46.487415-07:00 CRITICAL     /lib64/libpthread.so.0() [0x7f74d3e77000+0x7e65]
      

      Figuring out more details on the logs and test scenario. This is found by the scan logs/crashes utility.

      Attachments

        1. 172.23.105.211.tar.gz
          65.34 MB
        2. 172.23.105.212.tar.gz
          35.96 MB
        3. 172.23.105.213.tar.gz
          41.46 MB
        4. collectinfo-2020-11-12T050352-ns_1@172.23.122.136.zip
          22.52 MB
        5. collectinfo-2020-11-12T050352-ns_1@172.23.122.137.zip
          16.59 MB
        6. collectinfo-2020-11-12T050352-ns_1@172.23.122.148.zip
          12.39 MB
        7. collectinfo-2020-11-12T050352-ns_1@172.23.122.154.zip
          13.26 MB
        8. collectinfo-2020-11-12T050352-ns_1@172.23.122.155.zip
          13.15 MB
        9. collectinfo-2020-12-21T133322-ns_1@172.23.120.170.zip
          10.56 MB
        10. collectinfo-2020-12-21T133322-ns_1@172.23.121.115.zip
          7.06 MB
        11. collectinfo-2020-12-21T133322-ns_1@172.23.121.116.zip
          7.31 MB
        12. collectinfo-2020-12-21T133322-ns_1@172.23.121.123.zip
          7.21 MB
        13. collectinfo-2020-12-21T133322-ns_1@172.23.121.124.zip
          7.09 MB
        14. collectinfo-2020-12-21T133322-ns_1@172.23.121.126.zip
          7.12 MB
        15. collectinfo-2020-12-21T133322-ns_1@172.23.121.127.zip
          7.22 MB
        16. collectinfo-2020-12-21T133322-ns_1@172.23.121.128.zip
          6.68 MB
        17. collectinfo-2020-12-21T133322-ns_1@172.23.121.129.zip
          7.19 MB
        18. collectinfo-2020-12-21T133322-ns_1@172.23.121.130.zip
          7.13 MB
        19. collectinfo-2020-12-21T133322-ns_1@172.23.121.131.zip
          7.17 MB
        20. collectinfo-2020-12-21T133322-ns_1@172.23.121.132.zip
          7.13 MB
        21. collectinfo-2020-12-21T133322-ns_1@172.23.121.133.zip
          7.13 MB
        22. collectinfo-2020-12-21T133322-ns_1@172.23.121.134.zip
          7.13 MB
        23. collectinfo-2020-12-21T133322-ns_1@172.23.121.135.zip
          6.50 MB

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty