Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-39022

memcached crashed during crud.

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Critical
    • None
    • Cheshire-Cat
    • couchbase-bucket
    • 1006.5.1-1122
    • Untriaged
    • 1
    • Unknown

    Description

      172.23.105.225 : Stack Trace of 3437b692-366d-4c1c-281f365d-3f4d3c8f.dmp
      Core was generated by `/opt/couchbase/bin/memcached -C /opt/couchbase/var/lib/couchbase/config/memcach'.
       #0  0x00007f76c20ae207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
       55	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
       #0  0x00007f76c20ae207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
       #1  0x00007f76c20af8f8 in __GI_abort () at abort.c:90
       #2  0x00007f76c2c0b195 in __gnu_cxx::__verbose_terminate_handler () at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/vterminate.cc:95
       #3  0x0000000000545e02 in backtrace_terminate_handler() ()
       #4  0x00007f76c2c08f86 in __cxxabiv1::__terminate (handler=<optimized out>) at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
       #5  0x00007f76c2c08fd1 in std::terminate () at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
       #6  0x00007f76c2c09213 in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=0x7f3ab0 <typeinfo for std::logic_error>, dest=0x418bc0 <_ZNSt11logic_errorD1Ev@plt>) at /tmp/deploy/gcc-7.3.0/libstdc++-v3/libsupc++/eh_throw.cc:93
       #7  0x00007f76bd6b9b5d in Checkpoint::expelItems[abi:cxx11](CheckpointCursor&) () from /opt/couchbase/lib/ep.so
       #8  0x00007f76bd6c17b4 in CheckpointManager::expelUnreferencedCheckpointItems() () from /opt/couchbase/lib/ep.so
       #9  0x00007f76bd6cc5f4 in ClosedUnrefCheckpointRemoverTask::run() () from /opt/couchbase/lib/ep.so
       #10 0x00007f76bd792cc4 in ExecutorThread::run() () from /opt/couchbase/lib/ep.so
       #11 0x00007f76c4a2bee7 in platform_thread_wrap(void*) () from /opt/couchbase/lib/libplatform_so.so.0.1.0
       #12 0x00007f76c244cdd5 in start_thread (arg=0x7f75f87c6700) at pthread_create.c:307
       #13 0x00007f76c2175ead in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
      

      Please Note: Crash is seen during simple upserts and there is no kill memc involved in the test.

      Under the hood i see crash in magma:

      172.23.105.225 : Looking for CRITICAL messages in log
      172.23.105.225 : Found message in /opt/couchbase/var/lib/couchbase/logs/memcached.log.000139.txt
      2020-04-28T04:42:33.493535-07:00 CRITICAL WAL replay at offset 83:25165824, double BeginTxn seen.
      2020-04-28T04:42:33.495625-07:00 CRITICAL (default) magma_3 MagmaKVStore Magma open failed. Status:Corruption: Failed to initialize wal iterator :Invalid: WAL replay at offset 83:25165824, double BeginTxn seen.
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty