Details
-
Bug
-
Resolution: Fixed
-
Major
-
1.0.0-beta
-
None
-
1
Description
http://sdk.jenkins.couchbase.com/job/c-cpp/job/cxx/job/centos-cxx-sdk-server-situational-tests/78/
This was during a swap rebalance test
It looks like the mcbp_session was stopped which resets the session_ ptr to null in the message handler while a message was being processed. There can be a race between the check here https://github.com/couchbaselabs/couchbase-cxx-client/blob/9d329392687fec35f3fa91c95da655bd4a70dacb/couchbase/io/mcbp_session.hxx#L193, session_ being reset here https://github.com/couchbaselabs/couchbase-cxx-client/blob/9d329392687fec35f3fa91c95da655bd4a70dacb/couchbase/io/mcbp_session.hxx#L121 and session_ being used later in the message handler
13:24:17 Program terminated with signal SIGSEGV, Segmentation fault.
|
13:24:17 #0 0x0000000000cdecdb in std::__atomic_base<unsigned int>::operator++ (this=0x538) at /usr/include/c++/8/bits/atomic_base.h:296
|
13:24:17 296 { return __atomic_add_fetch(&_M_i, 1, memory_order_seq_cst); }
|
13:24:17 [Current thread is 1 (Thread 0x7fd01f6ba700 (LWP 30473))]
|
|
13:24:20 Thread 1 (Thread 0x7fd01f6ba700 (LWP 30473)):
|
13:24:20 #0 0x0000000000cdecdb in std::__atomic_base<unsigned int>::operator++ (this=0x538) at /usr/include/c++/8/bits/atomic_base.h:296
|
13:24:20 #1 0x0000000000cc6b52 in couchbase::io::mcbp_session::next_opaque (this=0x0) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/couchbase/io/mcbp_session.hxx:923
|
13:24:20 #2 0x0000000000cc0f8f in couchbase::io::mcbp_session::bootstrap_handler::handle (this=0x7fcf900044d0, msg=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/couchbase/io/mcbp_session.hxx:269
|
13:24:20 #3 0x0000000000cc91a8 in couchbase::io::mcbp_session::do_read()::{lambda(std::error_code, unsigned long)#1}::operator()(std::error_code, unsigned long) const (__closure=0x7fcffc0166a0, ec=..., bytes_transferred=345) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/couchbase/io/mcbp_session.hxx:1191
|
13:24:20 #4 0x0000000000cf320a in std::_Function_handler<void (std::error_code, unsigned long), couchbase::io::mcbp_session::do_read()::{lambda(std::error_code, unsigned long)#1}>::_M_invoke(std::_Any_data const&, std::error_code&&, unsigned long&&) (__functor=..., __args#0=..., __args#1=@0x7fd01f6b9170: 345) at /usr/include/c++/8/bits/std_function.h:297
|
13:24:20 #5 0x0000000000d2dd51 in std::function<void (std::error_code, unsigned long)>::operator()(std::error_code, unsigned long) const (this=0x7fd01f6b9300, __args#0=..., __args#1=345) at /usr/include/c++/8/bits/std_function.h:687
|
13:24:20 #6 0x0000000000d52f47 in asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>::operator()() (this=0x7fd01f6b9300) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/bind_handler.hpp:288
|
13:24:20 #7 0x0000000000d4bbad in asio::asio_handler_invoke<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, ...) (function=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/handler_invoke_hook.hpp:87
|
13:24:20 #8 0x0000000000d437b5 in asio_handler_invoke_helpers::invoke<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>, std::function<void (std::error_code, unsigned long)> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, std::function<void (std::error_code, unsigned long)>&) (function=..., context=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_invoke_helpers.hpp:54
|
13:24:20 #9 0x0000000000d73ed9 in asio::detail::asio_handler_invoke<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>, std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>*) (function=..., this_handler=0x7fd01f6b9300) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/bind_handler.hpp:343
|
13:24:20 #10 0x0000000000d7136e in asio_handler_invoke_helpers::invoke<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>, asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&) (function=..., context=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_invoke_helpers.hpp:54
|
13:24:20 #11 0x0000000000d6baf2 in asio::detail::executor_function::complete<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>, std::allocator<void> >(asio::detail::executor_function::impl_base*, bool) (base=0x7fcfdc005ce0, call=true) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/executor_function.hpp:115
|
13:24:20 #12 0x0000000000caeb07 in asio::detail::executor_function::operator() (this=0x7fd01f6b94d8) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/executor_function.hpp:63
|
13:24:20 #13 0x0000000000d5c1c1 in asio::asio_handler_invoke<asio::detail::executor_function> (function=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/handler_invoke_hook.hpp:87
|
13:24:20 #14 0x0000000000d53669 in asio_handler_invoke_helpers::invoke<asio::detail::executor_function, asio::detail::executor_function> (function=..., context=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_invoke_helpers.hpp:54
|
13:24:20 #15 0x0000000000d5c368 in asio::detail::executor_op<asio::detail::executor_function, std::allocator<void>, asio::detail::scheduler_operation>::do_complete (owner=0x7fd00c03d110, base=0x7fd00003d7e0) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/executor_op.hpp:69
|
13:24:20 #16 0x0000000000cafa4a in asio::detail::scheduler_operation::complete (this=0x7fd00003d7e0, owner=0x7fd00c03d110, ec=..., bytes_transferred=0) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/scheduler_operation.hpp:39
|
13:24:20 #17 0x0000000000cb8fe6 in asio::detail::strand_executor_service::run_ready_handlers (impl=std::shared_ptr<asio::detail::strand_executor_service::strand_impl> (use count 5, weak count 0) = {...}) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/strand_executor_service.ipp:149
|
13:24:20 #18 0x0000000000d783f3 in asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void>::operator() (this=0x7fd01f6b9710) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/strand_executor_service.hpp:126
|
13:24:20 #19 0x0000000000d77419 in asio::asio_handler_invoke<asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void> > (function=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/handler_invoke_hook.hpp:87
|
13:24:20 #20 0x0000000000d7566c in asio_handler_invoke_helpers::invoke<asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void>, asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void> > (function=..., context=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_invoke_helpers.hpp:54
|
13:24:20 #21 0x0000000000d73f64 in asio::io_context::basic_executor_type<std::allocator<void>, 4ul>::execute<asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void> > (this=0x7fd01f6b99e0, f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/impl/io_context.hpp:287
|
13:24:20 #22 0x0000000000d713f5 in asio_execution_execute_fn::impl::operator()<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const&, asio::detail::strand_executor_service::invoker<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, void> > (this=0x10442eb <asio_execution_execute_fn::static_instance<asio_execution_execute_fn::impl>::instance>, t=..., f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/execution/execute.hpp:204
|
13:24:20 #23 0x0000000000d6be80 in asio::detail::strand_executor_service::do_execute<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, asio::detail::executor_function, std::allocator<void> > (impl=std::shared_ptr<asio::detail::strand_executor_service::strand_impl> (use count 5, weak count 0) = {...}, ex=..., function=..., a=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/strand_executor_service.hpp:257
|
13:24:20 #24 0x0000000000d66033 in asio::detail::strand_executor_service::execute<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> const, asio::detail::executor_function> (impl=std::shared_ptr<asio::detail::strand_executor_service::strand_impl> (use count 5, weak count 0) = {...}, ex=..., function=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/strand_executor_service.hpp:208
|
13:24:20 #25 0x0000000000d5c0af in asio::strand<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> >::execute<asio::detail::executor_function> (this=0x7fd01f6b99e0, f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/strand.hpp:286
|
13:24:20 #26 0x0000000000d53503 in asio_execution_execute_fn::impl::operator()<asio::strand<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> > const&, asio::detail::executor_function> (this=0x10442eb <asio_execution_execute_fn::static_instance<asio_execution_execute_fn::impl>::instance>, t=..., f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/execution/execute.hpp:204
|
13:24:20 #27 0x0000000000d4c5a7 in asio::execution::detail::any_executor_base::execute_ex<asio::strand<asio::io_context::basic_executor_type<std::allocator<void>, 4ul> > > (ex=..., f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/execution/any_executor.hpp:893
|
13:24:20 #28 0x0000000000d5301b in asio::execution::detail::any_executor_base::execute<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&&) const (this=0x7fd01f6b99e0, f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/execution/any_executor.hpp:610
|
13:24:20 #29 0x0000000000d4bbd7 in asio_execution_execute_fn::impl::operator()<asio::any_io_executor, asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long> >(asio::any_io_executor&&, asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&&) const (this=0x10442eb <asio_execution_execute_fn::static_instance<asio_execution_execute_fn::impl>::instance>, t=..., f=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/execution/execute.hpp:204
|
13:24:20 #30 0x0000000000d437ff in asio::detail::handler_work_base<asio::any_io_executor, void, asio::io_context, asio::executor, void>::dispatch<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>, std::function<void (std::error_code, unsigned long)> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, std::function<void (std::error_code, unsigned long)>&) (this=0x7fd01f6b9ae0, function=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_work.hpp:438
|
13:24:20 #31 0x0000000000d3bd41 in asio::detail::handler_work<std::function<void (std::error_code, unsigned long)>, asio::any_io_executor, void>::complete<asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long> >(asio::detail::binder2<std::function<void (std::error_code, unsigned long)>, std::error_code, unsigned long>&, std::function<void (std::error_code, unsigned long)>&) (this=0x7fd01f6b9ae0, function=..., handler=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/handler_work.hpp:515
|
13:24:20 #32 0x0000000000d2e02f in asio::detail::reactive_socket_recv_op<asio::mutable_buffer, std::function<void (std::error_code, unsigned long)>, asio::any_io_executor>::do_complete(void*, asio::detail::scheduler_operation*, std::error_code const&, unsigned long) (owner=0x30c08d0, base=0x7fd00003d7e0) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/reactive_socket_recv_op.hpp:144
|
13:24:20 #33 0x0000000000cafa4a in asio::detail::scheduler_operation::complete (this=0x7fd00003d7e0, owner=0x30c08d0, ec=..., bytes_transferred=0) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/scheduler_operation.hpp:39
|
13:24:20 #34 0x0000000000cb2c5b in asio::detail::epoll_reactor::descriptor_state::do_complete (owner=0x30c08d0, base=0x7fcff8002820, ec=..., bytes_transferred=5) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/epoll_reactor.ipp:805
|
13:24:20 #35 0x0000000000cafa4a in asio::detail::scheduler_operation::complete (this=0x7fcff8002820, owner=0x30c08d0, ec=..., bytes_transferred=5) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/scheduler_operation.hpp:39
|
13:24:20 #36 0x0000000000cb3c8c in asio::detail::scheduler::do_run_one (this=0x30c08d0, lock=..., this_thread=..., ec=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/scheduler.ipp:491
|
13:24:20 #37 0x0000000000cb3610 in asio::detail::scheduler::run (this=0x30c08d0, ec=...) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/detail/impl/scheduler.ipp:209
|
13:24:20 #38 0x0000000000cb443b in asio::io_context::run (this=0x7ffedccca868) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/couchbase-cxx-client/third_party/asio/asio/include/asio/impl/io_context.ipp:62
|
13:24:20 #39 0x0000000000c9866f in CBSdkd::MainDispatch::<lambda()>::operator()(void) const (__closure=0x30e9208) at /home/couchbase/jenkins/workspace/c-cpp/cxx/centos-cxx-sdk-server-situational-tests/sdkd-cxx/src/Control.cpp:16
|
13:24:20 #40 0x0000000000c9bc85 in std::__invoke_impl<void, CBSdkd::MainDispatch::MainDispatch()::<lambda()> >(std::__invoke_other, CBSdkd::MainDispatch::<lambda()> &&) (__f=...) at /usr/include/c++/8/bits/invoke.h:60
|
13:24:20 #41 0x0000000000c9ad0e in std::__invoke<CBSdkd::MainDispatch::MainDispatch()::<lambda()> >(CBSdkd::MainDispatch::<lambda()> &&) (__fn=...) at /usr/include/c++/8/bits/invoke.h:95
|
13:24:20 #42 0x0000000000cacb9c in std::thread::_Invoker<std::tuple<CBSdkd::MainDispatch::MainDispatch()::<lambda()> > >::_M_invoke<0>(std::_Index_tuple<0>) (this=0x30e9208) at /usr/include/c++/8/thread:244
|
13:24:20 #43 0x0000000000cacb72 in std::thread::_Invoker<std::tuple<CBSdkd::MainDispatch::MainDispatch()::<lambda()> > >::operator()(void) (this=0x30e9208) at /usr/include/c++/8/thread:253
|
13:24:20 #44 0x0000000000cacb56 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<CBSdkd::MainDispatch::MainDispatch()::<lambda()> > > >::_M_run(void) (this=0x30e9200) at /usr/include/c++/8/thread:196
|
13:24:20 #45 0x00007fd02140fba3 in execute_native_thread_routine () from /lib64/libstdc++.so.6
|
13:24:20 #46 0x00007fd02248714a in start_thread () from /lib64/libpthread.so.0
|
13:24:20 #47 0x00007fd020aeadc3 in clone () from /lib64/libc.so.6
|
Attachments
Issue Links
- relates to
-
CXXCBC-450 Do not use bootstrap handler to change session state once re-bootstrap process has been started
- Closed