Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-47134

Underflow Exception from VBucket::doAggregatedFlushStats

    XMLWordPrintable

Details

    Description

      BackTraces:
      Core found on Node 172.23.121.130

      (gdb) bt full
      #0  0x00007f89ac66a387 in raise () from /lib64/libc.so.6
      No symbol table info available.
      #1  0x00007f89ac66ba78 in abort () from /lib64/libc.so.6
      No symbol table info available.
      #2  0x00007f89acfb563c in __gnu_cxx::__verbose_terminate_handler () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
              terminating = false
              t = <optimized out>
      #3  0x0000000000a9267b in backtrace_terminate_handler() ()
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/utilities/terminate_handler.cc:88
      No locals.
      #4  0x00007f89acfc08f6 in __cxxabiv1::__terminate(void (*)()) () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:48
      No locals.
      #5  0x00007f89acfc0961 in std::terminate () at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:58
      No locals.
       
       #6  0x00007f89acfc0bf4 in __cxxabiv1::__cxa_throw (obj=obj@entry=0x7f8968000940,
          tinfo=tinfo@entry=0xf95fe0 <typeinfo for boost::exception_detail::error_info_injector<std::underflow_error>>,
          dest=dest@entry=0x6dd410 <boost::exception_detail::error_info_injector<std::underflow_error>::~error_info_injector()>)
          at /tmp/deploy/objdir/../gcc-10.2.0/libstdc++-v3/libsupc++/eh_throw.cc:95
              globals = <optimized out>
              header = 0x7f89680008c0
      #7  0x00000000006e0cc2 in cb::throwWithTrace<std::underflow_error> (exception=...)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/tlm/deps/boost.exploded/include/boost/exception/info.hpp:129
      No locals.
      #8  0x00000000004a37da in cb::ThrowExceptionUnderflowPolicy<unsigned long>::underflow (current=140228509473056, arg=<optimized out>,
          desired=<optimized out>, this=<optimized out>)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:523
      No locals.
      #9  0x00000000007177d1 in fetch_sub (arg=<optimized out>, this=<optimized out>) at /opt/gcc-10.2.0/include/c++/10.2.0/bits/atomic_base.h:116
              current = <optimized out>
              desired = <optimized out>
      #10 decrDirtyQueuePendingWrites (decrementBy=<optimized out>, this=<optimized out>)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:3189
      No locals.
      #11 VBucket::doAggregatedFlushStats (this=this@entry=0x7f89286caa00, aggStats=...)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:715
              numItems = <optimized out>
      #12 0x00000000007e53a7 in EPBucket::flushSuccessEpilogue (this=this@entry=0x7f89a029e000, vb=..., flushStart=flushStart@entry=...,
          itemsFlushed=<optimized out>, itemsFlushed@entry=5863, aggStats=..., collectionFlush=...)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/ep_bucket.cc:860
      ---Type <return> to continue, or q <return> to quit---
              transTime = <optimized out>
      #13 0x00000000007ed1d7 in EPBucket::flushVBucket_UNLOCKED(LockedVBucketPtr) ()
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.h:2535
              vbid = {vbid = 31}
              toFlush = {
                items = {<std::_Vector_base<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> >, std::allocator<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> > > >> = {
                    _M_impl = {<std::allocator<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> > >> = {<__gnu_cxx::new_allocator<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> > >> = {<No data fields>}, <No data fields>}, <std::_Vector_base<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> >, std::allocator<SingleThreadedRCPtr<Item, Item*, std::default_delete<Item> > > >::_Vector_impl_data> = {_M_start = 0x0,
      

      QE-Test:

      guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/ankushj.ini bucket_storage=magma,rerun=false,bucket_eviction_policy=fullEviction,randomize_value=True,enable_dp=True,log_level=debug,infra_log_level=debug,GROUP=P0,copy_data=True -t storage.magma.magma_disk_full.MagmaDiskFull.test_deletes_disk_full,nodes_init=4,num_items=5000000,doc_size=2048,sdk_timeout=60,replicas=1,GROUP=P0'
      

      Attachments

        1. gdb.txt
          59 kB
        2. Thread_dump.docx
          32 kB
        3. TotalDataSize-48.png
          TotalDataSize-48.png
          28 kB
        For Gerrit Dashboard: MB-47134
        # Subject Branch Project Status CR V

        Activity

          People

            ankush.sharma Ankush Sharma
            ankush.sharma Ankush Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty