Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36087

DeleteWithMeta against a value evicted xattr document crashes memcached

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • master, 5.0.0, 5.1.0, 5.0.1, 5.1.1, 6.5.0, 6.0.0, 5.5.0, 5.5.1, 5.1.2, 5.5.2, 5.1.3, 5.5.3, 6.0.1, 6.0.2, 5.5.4, 5.5.5
    • 6.0.3
    • couchbase-bucket
    • Untriaged
    • Unknown

    Description

      Following steps will crash kv-engine

      1. set(k1, datatype=xattr)
      2. evict(k1)
      3. delete-with-meta(k1) -> crash

      This occurs because VBucket::deleteWithMeta has a clause that doesn't check if the value is present and proceeds to dereference the value to obtain the size, crash because the value is a null pointer.

      Following backtrace was seen:

      Linux (note trimmed the frames to include only final few).

      #0  std::__atomic_base<unsigned int>::load (__m=std::memory_order_seq_cst, this=<optimized out>) at /usr/include/c++/7/bits/atomic_base.h:396
      #1  std::__atomic_base<unsigned int>::operator unsigned int (this=<optimized out>) at /usr/include/c++/7/bits/atomic_base.h:259
      #2  Blob::valueSize (this=<optimized out>) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/blob.h:70
      #3  VBucket::pruneXattrDocument (this=this@entry=0x7f3c9e14ac00, v=..., itemMeta=...) at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:2759
      #4  0x00007f3d0d8fbd3a in VBucket::deleteWithMeta (this=0x7f3c9e14ac00, cas=@0x7f3d07a7bd78: 0, seqno=seqno@entry=0x7f3d07a7bd88, cookie=cookie@entry=0x7f3b2911d580, engine=..., bgFetchDelay=<optimized out>, checkConflicts=<optimized out>, itemMeta=..., backfill=<optimized out>, genBySeqno=<optimized out>, generateCas=<optimized out>, bySeqno=<optimized out>, isReplication=<optimized out>, readHandle=...)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/vbucket.cc:1505
      #5  0x00007f3d0d8c07e1 in KVBucket::deleteWithMeta (this=0x7f3d0152cb00, key=..., cas=@0x7f3d07a7bd78: 0, seqno=0x7f3d07a7bd88, vbucket=<optimized out>, cookie=0x7f3b2911d580, permittedVBStates=..., checkConflicts=CheckConflicts::Yes, itemMeta=..., backfill=false, genBySeqno=GenerateBySeqno::Yes, generateCas=GenerateCas::No, bySeqno=0, emd=0x0, isReplication=false)
          at /home/couchbase/jenkins/workspace/couchbase-server-unix/kv_engine/engines/ep/src/kv_bucket.cc:1817
      

      OSX (from unit test in http://review.couchbase.org/#/c/115076/1)

      * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x4)
        * frame #0: 0x0000000100188fd3 ep-engine_ep_unit_tests`VBucket::pruneXattrDocument(StoredValue&, ItemMetaData const&) [inlined] std::__1::__atomic_base<unsigned int, false>::load(this=0x0000000000000004, __m=memory_order_seq_cst) const at atomic:929 [opt]
          frame #1: 0x0000000100188fd3 ep-engine_ep_unit_tests`VBucket::pruneXattrDocument(StoredValue&, ItemMetaData const&) [inlined] std::__1::__atomic_base<unsigned int, false>::operator unsigned int(this=0x0000000000000004) const at atomic:933 [opt]
          frame #2: 0x0000000100188fd3 ep-engine_ep_unit_tests`VBucket::pruneXattrDocument(StoredValue&, ItemMetaData const&) [inlined] Blob::valueSize(this=0x0000000000000000) const at blob.h:68 [opt]
          frame #3: 0x0000000100188fd3 ep-engine_ep_unit_tests`VBucket::pruneXattrDocument(this=0x0000000104bc4600, v=0x0000000109166280, itemMeta=0x00007ffeefbff0c8) at vbucket.cc:3756 [opt]
          frame #4: 0x000000010018877d ep-engine_ep_unit_tests`VBucket::deleteWithMeta(this=0x0000000104bc4600, cas=0x00007ffeefbff238, seqno=0x00007ffeefbff220, cookie=0x0000000105ecce00, engine=0x0000000104bea400, checkConflicts=<unavailable>, itemMeta=0x00007ffeefbff0c8, genBySeqno=Yes, generateCas=No, bySeqno=0, cHandle=0x00007ffeefbfef60, deleteSource=Explicit) at vbucket.cc:2262 [opt]
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            jwalker Jim Walker added a comment - Test + possible fix - http://review.couchbase.org/#/c/115076/2

            Build couchbase-server-6.0.3-2884 contains kv_engine commit 893b9cd with commit message:
            MB-36087: deleteWithMeta was not always fetching non-resident items

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.0.3-2884 contains kv_engine commit 893b9cd with commit message: MB-36087 : deleteWithMeta was not always fetching non-resident items

            Build couchbase-server-6.5.0-4361 contains kv_engine commit 893b9cd with commit message:
            MB-36087: deleteWithMeta was not always fetching non-resident items

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.0-4361 contains kv_engine commit 893b9cd with commit message: MB-36087 : deleteWithMeta was not always fetching non-resident items

            Reopening for releasenote

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - Reopening for releasenote

            People

              jwalker Jim Walker
              jwalker Jim Walker
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty