Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-16366

Front-end operations (set/del) race with DCP snapshots

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 3.1.6, 4.1.2, 4.5.0
    • 3.0-Beta, 3.0, 3.0.1, 3.0.2, 3.0.3, 3.1.0, 4.0.0, 4.0.1, 4.1.0, 4.5.0
    • couchbase-bucket
    • Security Level: Public
    • None
    • Untriaged
    • No

    Description

      Taking SET as the example.

      SET against a replica VB is not allowed and the ep-engine code which drives the SET path first checks the VB state. If replica the operation is failed (NOT-MY-VB). However there is no exclusion with the entire SET path and the vbucket state changing to replica.

      Thus a SET can be underway at the same time as a DCP stream coming in and sending a snapshot, the snapshot then gets confused by the seqno being increased by the frontend operation triggering an assert in vbucket.h

      Assertion failed: (end >= (uint64_t)checkpointManager.getHighSeqno()), function setCurrentSnapshot_UNLOCKED, file /Users/build/3.03/ep-engine/src/vbucket.h, line 225.

      Vbucket state is checked at line 637, but could easily change state immediately after observing active.
      http://src.couchbase.org/source/xref/3.1.0/ep-engine/src/ep.cc#637

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MB-19323 3.1.6 release

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-16357 A race between compaction thread and the memcached thread that changes the vbucket state could fire an assertion crash

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ericcooper Eric Cooper (Inactive)
              jwalker Jim Walker
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty