Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-28733

ASan: heap-use-after-free in notify_io_complete

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 5.5.0
    • 5.5.0
    • memcached
    • None
    • Untriaged
    • Unknown

    Description

      As seen in CV when running test McdTestappTest.test_MB_16197/McbpSsl_JsonNo :

      ==15196==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110004f1948 at pc 0x00000052bba5 bp 0x7f413ec83120 sp 0x7f413ec83118
      READ of size 8 at 0x6110004f1948 thread T13 (ewb:pendingQ)
          #0 0x52bba4 in notify_io_complete(gsl::not_null<void const*>, ENGINE_ERROR_CODE) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/thread.cc:413
      ...
      0x6110004f1948 is located 72 bytes inside of 193-byte region [0x6110004f1900,0x6110004f19c1)
      freed by thread T1 here:
          #0 0x7f41609576b0 in operator delete(void*) (/usr/local/lib64/libasan.so.4+0xdb6b0)
      <cut>
          #7 0x7f415f5f229c in spdlog::details::async_log_helper::process_next_msg(std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:297
          #8 0x7f415f5f3069 in spdlog::details::async_log_helper::worker_loop() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:277
      

      Complete ASan report below. Note the line in question (thread.cc:413) was recently added in patch http://review.couchbase.org/#/c/90431/

      Console log: http://cv.jenkins.couchbase.com/job/kv_engine-addresssanitizer-master-gcc7/106/consoleFull#-207762811b106e815-6d27-4a01-8b14-4b94513c99be

      ==15196==ERROR: AddressSanitizer: heap-use-after-free on address 0x6110004f1948 at pc 0x00000052bba5 bp 0x7f413ec83120 sp 0x7f413ec83118
      READ of size 8 at 0x6110004f1948 thread T13 (ewb:pendingQ)
          #0 0x52bba4 in notify_io_complete(gsl::not_null<void const*>, ENGINE_ERROR_CODE) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/thread.cc:413
          #1 0x7f41450aceb5 in EWB_Engine::process_notifications() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/engines/ewouldblock_engine/ewouldblock_engine.cc:1942
          #2 0x7f415e2a46a0 in Couchbase::Thread::thread_entry() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/thread.cc:45
          #3 0x7f415e27c338 in CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:59
          #4 0x7f415e27c338 in platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:72
          #5 0x7f415c6596b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
          #6 0x7f415c38f41c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)
       
      0x6110004f1948 is located 72 bytes inside of 193-byte region [0x6110004f1900,0x6110004f19c1)
      freed by thread T1 here:
          #0 0x7f41609576b0 in operator delete(void*) (/usr/local/lib64/libasan.so.4+0xdb6b0)
          #1 0x7f415f5f229c in __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /usr/local/include/c++/7.3.0/ext/new_allocator.h:125
          #2 0x7f415f5f229c in std::allocator_traits<std::allocator<char> >::deallocate(std::allocator<char>&, char*, unsigned long) /usr/local/include/c++/7.3.0/bits/alloc_traits.h:462
          #3 0x7f415f5f229c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy(unsigned long) /usr/local/include/c++/7.3.0/bits/basic_string.h:226
          #4 0x7f415f5f229c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose() /usr/local/include/c++/7.3.0/bits/basic_string.h:221
          #5 0x7f415f5f229c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string() /usr/local/include/c++/7.3.0/bits/basic_string.h:647
          #6 0x7f415f5f229c in spdlog::details::async_log_helper::async_msg::~async_msg() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:57
          #7 0x7f415f5f229c in spdlog::details::async_log_helper::process_next_msg(std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&, std::chrono::time_point<std::chrono::_V2::system_clock, std::chrono::duration<long, std::ratio<1l, 1000000000l> > >&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:297
          #8 0x7f415f5f3069 in spdlog::details::async_log_helper::worker_loop() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:277
          #9 0x7f415f5e8af3 in void std::__invoke_impl<void, void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*>(std::__invoke_memfun_deref, void (spdlog::details::async_log_helper::*&&)(), spdlog::details::async_log_helper*&&) /usr/local/include/c++/7.3.0/bits/invoke.h:73
          #10 0x7f415f5e8af3 in std::__invoke_result<void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*>::type std::__invoke<void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*>(void (spdlog::details::async_log_helper::*&&)(), spdlog::details::async_log_helper*&&) /usr/local/include/c++/7.3.0/bits/invoke.h:95
          #11 0x7f415f5e8af3 in decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker<std::tuple<void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/local/include/c++/7.3.0/thread:234
          #12 0x7f415f5e8af3 in std::thread::_Invoker<std::tuple<void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*> >::operator()() /usr/local/include/c++/7.3.0/thread:243
          #13 0x7f415f5e8af3 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (spdlog::details::async_log_helper::*)(), spdlog::details::async_log_helper*> > >::_M_run() /usr/local/include/c++/7.3.0/thread:186
          #14 0x7f415ce4795e in execute_native_thread_routine /tmp/deploy/objdir/../gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc:83
       
      previously allocated by thread T3 (mc:worker_0) here:
          #0 0x7f41609569b0 in operator new(unsigned long) (/usr/local/lib64/libasan.so.4+0xda9b0)
          #1 0x7f415f5de6d8 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) /usr/local/include/c++/7.3.0/bits/basic_string.tcc:219
          #2 0x7f415f5eeb52 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char const*>(char const*, char const*, std::__false_type) /usr/local/include/c++/7.3.0/bits/basic_string.h:236
          #3 0x7f415f5eeb52 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*) /usr/local/include/c++/7.3.0/bits/basic_string.h:255
          #4 0x7f415f5eeb52 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) /usr/local/include/c++/7.3.0/bits/basic_string.h:502
          #5 0x7f415f5eeb52 in spdlog::details::async_log_helper::async_msg::async_msg(spdlog::details::log_msg const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:100
          #6 0x7f415f5eeb52 in spdlog::details::async_log_helper::log(spdlog::details::log_msg const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_log_helper.h:241
          #7 0x7f415f5eeb52 in spdlog::async_logger::_sink_it(spdlog::details::log_msg&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/async_logger_impl.h:93
          #8 0x5bacd8 in void spdlog::logger::log<unsigned int, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(spdlog::level::level_enum, char const*, unsigned int const&, char const* const&, unsigned int const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/logger_impl.h:71
          #9 0x5bacd8 in Cookie::maybeLogSlowCommand(std::chrono::duration<long, std::ratio<1l, 1000000000l> > const&) const /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/cookie.cc:468
          #10 0x5dca3f in mcbp_collect_timings(Cookie&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/mcbp.cc:254
          #11 0x6ba094 in conn_execute /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/statemachine_mcbp.cc:365
          #12 0x59990b in McbpConnection::runStateMachinery() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connection_mcbp.cc:792
          #13 0x599e23 in McbpConnection::runEventLoop(short) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connection_mcbp.cc:973
          #14 0x5a4e8d in run_event_loop(Connection*, short) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connections.cc:159
          #15 0x48a232 in event_handler(int, short, void*) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:1018
          #16 0x7f415dc3aa23 in event_process_active_single_queue.isra.26 (/home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/install/lib/libevent_core.so.2.1.8+0x1aa23)
       
      Thread T13 (ewb:pendingQ) created by T12 (mc:bucket_add) here:
          #0 0x7f41608b2a00 in __interceptor_pthread_create (/usr/local/lib64/libasan.so.4+0x36a00)
          #1 0x7f415e27be74 in cb_create_named_thread /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:110
          #2 0x7f415e2a4aa5 in Couchbase::Thread::start() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/thread.cc:72
          #3 0x7f41450a6a1b in EWB_Engine::EWB_Engine(server_handle_v1_t* (*)()) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/engines/ewouldblock_engine/ewouldblock_engine.cc:1389
          #4 0x7f41450a7668 in create_instance /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/engines/ewouldblock_engine/ewouldblock_engine.cc:1878
          #5 0x7f415f886a55 in create_engine_instance(engine_reference*, server_handle_v1_t* (*)(), engine_interface**) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/utilities/engine_loader.cc:114
          #6 0x5c105e in Engine::createInstance(server_handle_v1_t* (*)(), engine_interface**) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/enginemap.cc:40
          #7 0x5c105e in new_engine_instance /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/enginemap.cc:76
          #8 0x48bd80 in CreateBucketThread::create() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:1915
          #9 0x48d39e in CreateBucketThread::run() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:1991
          #10 0x7f415e2a46a0 in Couchbase::Thread::thread_entry() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/thread.cc:45
          #11 0x7f415e27c338 in CouchbaseThread::run() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:59
          #12 0x7f415e27c338 in platform_thread_wrap /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:72
          #13 0x7f415c6596b9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
       
      Thread T12 (mc:bucket_add) created by T3 (mc:worker_0) here:
          #0 0x7f41608b2a00 in __interceptor_pthread_create (/usr/local/lib64/libasan.so.4+0x36a00)
          #1 0x7f415e27be74 in cb_create_named_thread /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:110
          #2 0x7f415e2a4aa5 in Couchbase::Thread::start() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/thread.cc:72
          #3 0x632387 in McbpCreateBucketTask::start() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/create_remove_bucket_command_context.cc:39
          #4 0x632387 in CreateRemoveBucketCommandContext::create() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/create_remove_bucket_command_context.cc:83
          #5 0x634e9e in CreateRemoveBucketCommandContext::step() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/create_remove_bucket_command_context.cc:126
          #6 0x6a7fcb in SteppableCommandContext::drive() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/protocol/mcbp/steppable_command_context.cc:33
          #7 0x5ec644 in create_remove_bucket_executor /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:470
          #8 0x5f6c07 in std::function<void (Cookie&)>::operator()(Cookie&) const /usr/local/include/c++/7.3.0/bits/std_function.h:706
          #9 0x5f6c07 in execute_request_packet /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:743
          #10 0x5f6c07 in mcbp_execute_packet(Cookie&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/mcbp_executors.cc:824
          #11 0x6ba03c in conn_execute /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/statemachine_mcbp.cc:350
          #12 0x59990b in McbpConnection::runStateMachinery() /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connection_mcbp.cc:792
          #13 0x599e23 in McbpConnection::runEventLoop(short) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connection_mcbp.cc:973
          #14 0x5a4e8d in run_event_loop(Connection*, short) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/connections.cc:159
          #15 0x48a232 in event_handler(int, short, void*) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:1018
          #16 0x7f415dc3aa23 in event_process_active_single_queue.isra.26 (/home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/install/lib/libevent_core.so.2.1.8+0x1aa23)
       
      Thread T3 (mc:worker_0) created by T0 here:
          #0 0x7f41608b2a00 in __interceptor_pthread_create (/usr/local/lib64/libasan.so.4+0x36a00)
          #1 0x7f415e27be74 in cb_create_named_thread /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/platform/src/cb_pthreads.cc:110
          #2 0x52f864 in create_worker /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/thread.cc:86
          #3 0x52f864 in thread_init(unsigned long, event_base*, void (*)(int, short, void*)) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/thread.cc:515
          #4 0x498a10 in memcached_main /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:2548
          #5 0x7f415c2a882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
       
      Thread T1 created by T0 here:
          #0 0x7f41608b2a00 in __interceptor_pthread_create (/usr/local/lib64/libasan.so.4+0x36a00)
          #1 0x7f415ce47c14 in __gthread_create /tmp/deploy/objdir/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:662
          #2 0x7f415ce47c14 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /tmp/deploy/objdir/../gcc-7.3.0/libstdc++-v3/src/c++11/thread.cc:163
          #3 0x7f415f5e3899 in void __gnu_cxx::new_allocator<spdlog::async_logger>::construct<spdlog::async_logger, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(spdlog::async_logger*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/ext/new_allocator.h:136
          #4 0x7f415f5e3899 in void std::allocator_traits<std::allocator<spdlog::async_logger> >::construct<spdlog::async_logger, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::allocator<spdlog::async_logger>&, spdlog::async_logger*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/alloc_traits.h:475
          #5 0x7f415f5e3899 in std::_Sp_counted_ptr_inplace<spdlog::async_logger, std::allocator<spdlog::async_logger>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::allocator<spdlog::async_logger>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr_base.h:526
          #6 0x7f415f5e3899 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<spdlog::async_logger, std::allocator<spdlog::async_logger>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::_Sp_make_shared_tag, spdlog::async_logger*, std::allocator<spdlog::async_logger> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr_base.h:637
          #7 0x7f415f5e3899 in std::__shared_ptr<spdlog::async_logger, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<spdlog::async_logger>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::_Sp_make_shared_tag, std::allocator<spdlog::async_logger> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr_base.h:1295
          #8 0x7f415f5e3899 in std::shared_ptr<spdlog::async_logger>::shared_ptr<std::allocator<spdlog::async_logger>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::_Sp_make_shared_tag, std::allocator<spdlog::async_logger> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr.h:344
          #9 0x7f415f5e3899 in std::shared_ptr<spdlog::async_logger> std::allocate_shared<spdlog::async_logger, std::allocator<spdlog::async_logger>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::allocator<spdlog::async_logger> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr.h:691
          #10 0x7f415f5e3899 in std::shared_ptr<spdlog::async_logger> std::make_shared<spdlog::async_logger, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&, unsigned long&, spdlog::async_overflow_policy const&, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /usr/local/include/c++/7.3.0/bits/shared_ptr.h:707
          #11 0x7f415f5e3899 in std::shared_ptr<spdlog::async_logger> spdlog::details::registry_t<std::mutex>::create_async<std::shared_ptr<spdlog::sinks::sink> const*>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, spdlog::async_overflow_policy, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&, std::shared_ptr<spdlog::sinks::sink> const* const&, std::shared_ptr<spdlog::sinks::sink> const* const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/registry.h:79
          #12 0x7f415f5e3899 in spdlog::details::registry_t<std::mutex>::create_async(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, spdlog::async_overflow_policy, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&, std::initializer_list<std::shared_ptr<spdlog::sinks::sink> >) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/registry.h:124
          #13 0x7f415f5e3899 in spdlog::details::registry_t<std::mutex>::create_async(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, spdlog::async_overflow_policy, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&, std::shared_ptr<spdlog::sinks::sink>) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/registry.h:129
          #14 0x7f415f5e3899 in spdlog::create_async(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<spdlog::sinks::sink> const&, unsigned long, spdlog::async_overflow_policy, std::function<void ()> const&, std::chrono::duration<long, std::ratio<1l, 1000l> > const&, std::function<void ()> const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/third_party/spdlog/include/spdlog/details/spdlog_impl.h:209
          #15 0x7f415f5e3899 in cb::logger::initialize[abi:cxx11](cb::logger::Config const&) /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/logger/spdlogger.cc:202
          #16 0x497d6a in memcached_main /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/memcached.cc:2456
          #17 0x7f415c2a882f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
       
      SUMMARY: AddressSanitizer: heap-use-after-free /home/couchbase/jenkins/workspace/kv_engine-addresssanitizer-master-gcc7/kv_engine/daemon/thread.cc:413 in notify_io_complete(gsl::not_null<void const*>, ENGINE_ERROR_CODE)
      Shadow bytes around the buggy address:
        0x0c22800962d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c22800962e0: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa
        0x0c22800962f0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
        0x0c2280096300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c2280096310: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      =>0x0c2280096320: fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd
        0x0c2280096330: fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa
        0x0c2280096340: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
        0x0c2280096350: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
        0x0c2280096360: fd fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
        0x0c2280096370: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      Shadow byte legend (one shadow byte represents 8 application bytes):
        Addressable:           00
        Partially addressable: 01 02 03 04 05 06 07 
        Heap left redzone:       fa
        Freed heap region:       fd
        Stack left redzone:      f1
        Stack mid redzone:       f2
        Stack right redzone:     f3
        Stack after return:      f5
        Stack use after scope:   f8
        Global redzone:          f9
        Global init order:       f6
        Poisoned by user:        f7
        Container overflow:      fc
        Array cookie:            ac
        Intra object redzone:    bb
        ASan internal:           fe
        Left alloca redzone:     ca
        Right alloca redzone:    cb
      ==15196==ABORTING
      

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-28733
          # Subject Branch Project Status CR V

          Activity

            People

              trond Trond Norbye
              drigby Dave Rigby (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty