Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-29480

Cursor dropping doesn't respect the purge-seqno

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 4.6.4, 5.1.0, 5.5.0
    • Fix Version/s: 5.5.0
    • Component/s: couchbase-bucket
    • Labels:
    • Triage:
      Untriaged
    • Is this a Regression?:
      Unknown

      Description

      The following sequence demonstrates the issue (this has been performed via a unit-test)

      • Create DCP and stream from vb0
      • store into vb0 k1,k2,k3,k4 and k5
      • stream all items, client knows about k1,k2,k3,k4 and k5
      • delete k3 and k4 (we have to delete a minimum of 2 items because compaction never purges a delete if its the last seqno)
      • run compaction - k3 tombstone is purged
      • stream is slow... so we drop its cursor and go to backfill mode
      • backfill finds delete(k4)
      • client streams all items available and gets delete(k4)

      Database has k1,k2,k5
      Client has k1,k2,k3,k5

      The cursor drop backfill was allowed to go beyond the purge-seqno, a normal stream request with backfill would be a rollback situation.

        Attachments

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            jwalker Jim Walker created issue -
            jwalker Jim Walker made changes -
            Field Original Value New Value
            Description The following sequence demonstrates the issue (this has been performed via a unit-test)

            * Create DCP and stream from vb0
            * store into vb0 k1,k2,k3,k4 and k5
            * stream all items, client knows about k1,k2,k3,k4 and k5
            * delete k3 and k4 (we have to delete a minimum of 2 items because compaction never purges a delete if its the last seqno)
            * run compaction - k3 tombstone is purged
            * stream is slow... so we drop its cursor and go to backfill mode
            * backfill finds delete(k4)
            * client streams all items available and gets delete(k4)

            Database has k1,k2,k5
            Client has k1,k2,k3,k5

            The cursor drop backfill was allowed to go beyond the purge-seqno, a normal stream request with backfill would be a rollback situation.
            jwalker Jim Walker made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            jwalker Jim Walker made changes -
            Affects Version/s 5.1.0 [ 14611 ]
            Affects Version/s 4.6.4 [ 14607 ]
            Affects Version/s vulcan [ 14610 ]
            drigby Dave Rigby made changes -
            Link This issue relates to MB-29369 [ MB-29369 ]
            drigby Dave Rigby made changes -
            Fix Version/s vulcan [ 14610 ]
            drigby Dave Rigby made changes -
            Link This issue relates to MB-29512 [ MB-29512 ]
            jwalker Jim Walker made changes -
            Labels dataloss
            paolo.cocchi Paolo Cocchi [X] (Inactive) made changes -
            Link This issue is triggering MB-29514 [ MB-29514 ]
            jwalker Jim Walker made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-5.5.0-2685 contains kv_engine commit de9e025dc2978997769e456f4c78fe30feb60206 with commit message:
            MB-29480, MB-29512: Fail backfills that go below purge-seqno
            https://github.com/couchbase/kv_engine/commit/de9e025dc2978997769e456f4c78fe30feb60206

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-5.5.0-2685 contains kv_engine commit de9e025dc2978997769e456f4c78fe30feb60206 with commit message: MB-29480 , MB-29512 : Fail backfills that go below purge-seqno https://github.com/couchbase/kv_engine/commit/de9e025dc2978997769e456f4c78fe30feb60206
            Hide
            build-team Couchbase Build Team added a comment -

            Build couchbase-server-6.0.0-1110 contains kv_engine commit de9e025dc2978997769e456f4c78fe30feb60206 with commit message:
            MB-29480, MB-29512: Fail backfills that go below purge-seqno
            https://github.com/couchbase/kv_engine/commit/de9e025dc2978997769e456f4c78fe30feb60206

            Show
            build-team Couchbase Build Team added a comment - Build couchbase-server-6.0.0-1110 contains kv_engine commit de9e025dc2978997769e456f4c78fe30feb60206 with commit message: MB-29480 , MB-29512 : Fail backfills that go below purge-seqno https://github.com/couchbase/kv_engine/commit/de9e025dc2978997769e456f4c78fe30feb60206
            jwalker Jim Walker made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            jwalker Jim Walker made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            drigby Dave Rigby made changes -
            Resolution Fixed [ 1 ]
            Status Closed [ 6 ] Reopened [ 4 ]
            drigby Dave Rigby made changes -
            Security Private [ 10010 ]
            drigby Dave Rigby made changes -
            Resolution Fixed [ 1 ]
            Status Reopened [ 4 ] Closed [ 6 ]

              People

              • Assignee:
                jwalker Jim Walker
                Reporter:
                jwalker Jim Walker
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty

                    Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.