Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45899

Crash observed in Collections::Manager::doCollectionStats at kv_engine/engines/ep/src/collections/manager.cc:803

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 7.0.0
    • Cheshire-Cat
    • couchbase-bucket
    • 7.0.0-5017
    • Untriaged
    • 1
    • Yes
    • KV-Engine CC Final Sprint

    Description

      Crash Dump on node 172.23.123.102

      Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'.
      #0  operator* (this=<synthetic pointer>) at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/vbucket_manifest.cc:1125
      1125	/tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/vbucket_manifest.cc: No such file or directory.
      warning: File "/opt/couchbase/lib/libstdc++.so.6.0.28-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py:/usr/lib/golang/src/pkg/runtime/runtime-gdb.py".
      #0  operator* (this=<synthetic pointer>) at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/vbucket_manifest.cc:1125
      #1  operator-> (this=<synthetic pointer>) at /tmp/workspace/couchbase-server-unix/server_build/tlm/deps/folly.exploded/include/folly/container/detail/F14Policy.h:719
      #2  Collections::VB::Manifest::accumulateStats (this=0x7faa664f6e40, collections=..., summary=...) at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/vbucket_manifest.cc:1125
      #3  0x000000000074be3c in accumulateStats (summary=..., collections=..., this=0x7faac67e9930) at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/vbucket_manifest_handles.h:182
      #4  CollectionsGetStatsVBucketVisitor::visitBucket (this=0x7faac67e9a80, vb=...) at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/manager.cc:390
      #5  0x00000000006aa03b in KVBucket::visit(VBucketVisitor&) () at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/kv_bucket.cc:2253
      #6  0x00000000007448f0 in Collections::Manager::getPerCollectionStats(std::vector<Collections::CollectionEntry, std::allocator<Collections::CollectionEntry> > const&, KVBucket&) () at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/manager.cc:803
      #7  0x0000000000748dbb in Collections::Manager::doOneCollectionStats(KVBucket&, BucketStatCollector const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () at /opt/gcc-10.2.0/include/c++/10.2.0/bits/stl_uninitialized.h:90
      #8  0x0000000000749352 in Collections::Manager::doCollectionStats(KVBucket&, BucketStatCollector const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/collections/manager.cc:463
      #9  0x000000000066d712 in EventuallyPersistentEngine::doCollectionStats(void const*, std::function<void (std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, gsl::not_null<void const*>)> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_engine.cc:4351
      #10 0x000000000066f48b in EventuallyPersistentEngine::getStats(void const*, std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, std::function<void (std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, gsl::not_null<void const*>)> const&) () at /tmp/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_engine.cc:4762
      #11 0x000000000066f6f0 in EventuallyPersistentEngine::get_stats(gsl::not_null<void const*>, std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, std::function<void (std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, gsl::not_null<void const*>)> const&) () at /tmp/workspace/couchbase-server-unix/third_party/gsl-lite/include/gsl/gsl-lite.hpp:673
      #12 0x0000000000565017 in bucket_get_stats(Cookie&, std::basic_string_view<char, std::char_traits<char> >, cb::sized_buffer<unsigned char const>, std::function<void (std::basic_string_view<char, std::char_traits<char> >, std::basic_string_view<char, std::char_traits<char> >, gsl::not_null<void const*>)> const&) () at /tmp/workspace/couchbase-server-unix/third_party/gsl-lite/include/gsl/gsl-lite.hpp:673
      #13 0x00000000006029d3 in StatsCommandContext::doStats() () at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/protocol/mcbp/stats_context.cc:721
      #14 0x0000000000602cd8 in StatsCommandContext::step (this=0x7faaceb61750) at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/protocol/mcbp/stats_context.cc:643
      #15 0x0000000000606c90 in SteppableCommandContext::drive() () at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/protocol/mcbp/steppable_command_context.cc:36
      #16 0x00000000005c4bd8 in Cookie::execute(bool) () at /tmp/workspace/couchbase-server-unix/kv_engine/include/mcbp/protocol/header.h:110
      #17 0x00000000005aa018 in Connection::executeCommandPipeline() () at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/cookie.h:466
      #18 0x00000000005ab847 in Connection::executeCommandsCallback() () at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/connection.cc:688
      #19 0x00000000005ac461 in Connection::rw_callback(bufferevent*, void*) () at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/connection.cc:885
      #20 0x00007faad1e90d8e in bufferevent_run_deferred_callbacks_unlocked (cb=<optimized out>, arg=0x7faaceb70e80) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/bufferevent.c:208
      #21 0x00007faad1e99d01 in event_process_active_single_queue (base=base@entry=0x7faaceca5300, max_to_process=max_to_process@entry=2147483647, endtime=endtime@entry=0x0, activeq=<optimized out>) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1726
      #22 0x00007faad1e9a54f in event_process_active (base=0x7faaceca5300) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:1789
      #23 event_base_loop (base=0x7faaceca5300, flags=flags@entry=0) at /home/couchbase/jenkins/workspace/cbdeps-platform-build-old/deps/packages/build/libevent/libevent-prefix/src/libevent/event.c:2012
      #24 0x000000000058609b in worker_libevent (arg=0x7faacecf9000) at /tmp/workspace/couchbase-server-unix/kv_engine/daemon/thread.cc:202
      #25 0x00007faad2be7bd9 in run (this=0x7faacec56c20) at /tmp/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:58
      #26 platform_thread_wrap(void*) () at /tmp/workspace/couchbase-server-unix/platform/src/cb_pthreads.cc:71
      #27 0x00007faad05bbe65 in start_thread () from /lib64/libpthread.so.0
      #28 0x00007faad02e488d in hcreate_r () from /lib64/libc.so.6
      #29 0x0000000000000000 in ?? ()
      

      QE Test

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/testexec.13993.ini GROUP=P2,threads_to_use=10,upgrade_version=7.0.0-5017 -t cbas.cbas_collection_functional_test.CBASDatasetsAndCollections.test_KV_collection_deletion_does_not_effect_dataset_on_other_collections,bucket_spec=analytics.single_bucket,index_name=idx_age,index_fields=age:bigint,cardinality=1,bucket_cardinality=3,analytics_index=False,nodes_init=4,services_init=kv:n1ql-kv-cbas-cbas,GROUP=P2'
      

      Here is the job link where we saw the failure: http://cb-logs-qe.s3-website-us-west-2.amazonaws.com/7.0.0-5017/jenkins_logs/test_suite_executor-TAF/109241/consoleText.txt
      Last Passed build: 4960

      Attachments

        For Gerrit Dashboard: MB-45899
        # Subject Branch Project Status CR V

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty