Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42657

TSan: signal-unsafe call inside of a signal during Breakpad crash-catch

    XMLWordPrintable

Details

    • Triaged
    • 1
    • Unknown
    • KV-Engine Sprint 2021 July

    Description

      During local testing of FollyExecutorPool under TSan running make simple-test the following error was reported by TSan after Breakpad reported an issue with phosphor:

      WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=18845)
      

      Full backtrace below.

      To clarify, this MB is tracking the TSan issue after a crash is seem - the actual crash it it's own bug, which is tracked via MB-42441.

      WARNING: ThreadSanitizer: signal-unsafe call inside of a signal (pid=18845)
          #0 operator new(unsigned long) <null> (libtsan.so.0+0x00000006d926)
          #1 __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/ext/new_allocator.h:111 (libmemcached_logger.so.1.0.0+0x000000012752)
          #2 std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/alloc_traits.h:436 (libmemcached_logger.so.1.0.0+0x000000012752)
          #3 std::allocator<char>::value_type* fmt::v5::internal::allocate<std::allocator<char> >(std::allocator<char>&, unsigned long) /home/couchbase/server/third_party/spdlog/include/spdlog/fmt/bundled/format.h:403 (libmemcached_logger.so.1.0.0+0x000000012752)
          #4 fmt::v5::basic_memory_buffer<char, 176ul, std::allocator<char> >::grow(unsigned long) /home/couchbase/server/third_party/spdlog/include/spdlog/fmt/bundled/format.h:710 (libmemcached_logger.so.1.0.0+0x000000012752)
          #5 fmt::v5::internal::basic_buffer<char>::reserve(unsigned long) /home/couchbase/server/third_party/spdlog/include/spdlog/fmt/bundled/core.h:431 (libmemcached_logger.so.1.0.0+0x00000001253e)
          #6 void fmt::v5::internal::basic_buffer<char>::append<char>(char const*, char const*) /home/couchbase/server/third_party/spdlog/include/spdlog/fmt/bundled/format.h:566 (libmemcached_logger.so.1.0.0+0x00000001253e)
          #7 void spdlog::details::fmt_helper::append_buf<500ul, 176ul>(fmt::v5::basic_memory_buffer<char, 500ul, std::allocator<char> > const&, fmt::v5::basic_memory_buffer<char, 176ul, std::allocator<char> >&) /home/couchbase/server/third_party/spdlog/include/spdlog/details/fmt_helper.h:37 (libmemcached_logger.so.1.0.0+0x00000001253e)
          #8 async_msg /home/couchbase/server/third_party/spdlog/include/spdlog/details/thread_pool.h:80 (libmemcached_logger.so.1.0.0+0x00000001253e)
          #9 spdlog::details::thread_pool::post_log(std::shared_ptr<spdlog::async_logger>&&, spdlog::details::log_msg&&, spdlog::async_overflow_policy) /home/couchbase/server/third_party/spdlog/include/spdlog/details/thread_pool.h:151 (libmemcached_logger.so.1.0.0+0x00000000fe57)
          #10 spdlog::async_logger::sink_it_(spdlog::details::log_msg&) /home/couchbase/server/third_party/spdlog/include/spdlog/details/async_logger_impl.h:46 (libmemcached_logger.so.1.0.0+0x00000000f892)
          #11 void spdlog::logger::log<char [11], char const*>(spdlog::level::level_enum, char const*, char const (&) [11], char const* const&) /home/couchbase/server/third_party/spdlog/include/spdlog/details/logger_impl.h:65 (memcached+0x0000006702af)
          #12 dumpCallback(google_breakpad::MinidumpDescriptor const&, void*, bool) /home/couchbase/server/kv_engine/utilities/breakpad.cc:124 (memcached+0x00000066fc4c)
          #13 google_breakpad::ExceptionHandler::GenerateDump(google_breakpad::ExceptionHandler::CrashContext*) src/client/linux/handler/exception_handler.cc:566 (memcached+0x000000683d79)
          #14 std::atomic<phosphor::ChunkLock::State>::compare_exchange_weak(phosphor::ChunkLock::State&, phosphor::ChunkLock::State, std::memory_order, std::memory_order) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/atomic:288 (libphosphor.so+0x00000000e8a9)
          #15 std::atomic<phosphor::ChunkLock::State>::compare_exchange_weak(phosphor::ChunkLock::State&, phosphor::ChunkLock::State, std::memory_order) /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/atomic:307 (libphosphor.so+0x00000000e8a9)
          #16 phosphor::ChunkLock::lockMaster() /home/couchbase/server/phosphor/src/chunk_lock.cc:72 (libphosphor.so+0x00000000e8a9)
          #17 phosphor::MasterChunkLock::lock() /home/couchbase/server/phosphor/src/chunk_lock.cc:105 (libphosphor.so+0x00000000e8a9)
          #18 phosphor::TraceLog::evictThreads(std::lock_guard<phosphor::TraceLog>&) /home/couchbase/server/phosphor/src/trace_log.cc:309 (libphosphor.so+0x0000000193fc)
          #19 phosphor::TraceLog::stop(std::lock_guard<phosphor::TraceLog>&, bool) /home/couchbase/server/phosphor/src/trace_log.cc:93 (libphosphor.so+0x0000000193fc)
          #20 phosphor::TraceLog::stop(bool) /home/couchbase/server/phosphor/src/trace_log.cc:70 (libphosphor.so+0x000000018cae)
          #21 deinitializeTracing() /home/couchbase/server/kv_engine/daemon/tracing.cc:107 (memcached+0x0000004fb6b6)
          #22 memcached_main(int, char**) /home/couchbase/server/kv_engine/daemon/memcached.cc:1563 (memcached+0x00000046d4e0)
          #23 main /home/couchbase/server/kv_engine/daemon/main.cc:36 (memcached+0x0000004659a0)
      

      kv_engine commit: http://review.couchbase.org/c/kv_engine/+/135709/20

      Complete log: sanitizers.log.18845

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              drigby Dave Rigby (Inactive)
              drigby Dave Rigby (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty