Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-32227

Improve dcp rollback handling

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 6.0.0
    • 6.5.0
    • secondary-index

    Description

      Indexer's handling of dcp rollback can be improved:

      1. If DCP instructs to rollback to 0, indexer ignores the disk snapshots and blindly rollbacks to 0. There may be cases where trying the stream request with disk snapshots may succeed (e.g. only vbuuid has changed). It is better to always exhaust trying dcp stream requests with all the disk snapshots before rolling back to 0.

      See http://review.couchbase.org/#/c/74784/ for reference.

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-32227
          # Subject Branch Project Status CR V

          Activity

            Closing based on the verification of above tests done with 6.5.0-4789 and tests verified in : https://docs.google.com/spreadsheets/d/11vNDoQFV75YD71zcYnZ7ExxXaFvV9itfJIENUMhIpDI/edit#gid=0

            girish.benakappa Girish Benakappa added a comment - Closing based on the verification of above tests done with 6.5.0-4789 and tests verified in : https://docs.google.com/spreadsheets/d/11vNDoQFV75YD71zcYnZ7ExxXaFvV9itfJIENUMhIpDI/edit#gid=0

            Build couchbase-server-5.5.4-4313 contains indexing commit c964d93 with commit message:
            MB-32640 Retry stream request with all disk snapshots...

            build-team Couchbase Build Team added a comment - Build couchbase-server-5.5.4-4313 contains indexing commit c964d93 with commit message: MB-32640 Retry stream request with all disk snapshots...

            Build couchbase-server-6.0.1-1993 contains indexing commit cc88bd7 with commit message:
            MB-31989 Retry stream request with all disk snapshots...

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.0.1-1993 contains indexing commit cc88bd7 with commit message: MB-31989 Retry stream request with all disk snapshots...

            Build couchbase-server-6.5.0-1818 contains indexing commit 0f1b4ee with commit message:
            MB-32227 Retry stream request with all disk snapshots...

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.0-1818 contains indexing commit 0f1b4ee with commit message: MB-32227 Retry stream request with all disk snapshots...

            QE verification steps:

            1. Create a bucket with 2 replicas.
            2. Load data and achieve 20% resident ratio in the bucket.
            3. Create few indexes. Reduce disk snapshot interval to 1 min.

            curl -X POST -u Administrator:asdasd http://localhost:9102/settings --data '{"indexer.settings.persisted_snapshot.moi.interval":60000}'
            

            5. Do more mutations for a few minutes to create a couple of disk snapshots.
            6. While there is incoming data load in memcached, kill memcached on one node and failover the node in quick succession.
            7. Failover shouldn't cause indexes to rollback to 0 and then rebuild again.

            Also, it would be good to run all rollback related tests to make sure there is no regression.

            deepkaran.salooja Deepkaran Salooja added a comment - QE verification steps: 1. Create a bucket with 2 replicas. 2. Load data and achieve 20% resident ratio in the bucket. 3. Create few indexes. Reduce disk snapshot interval to 1 min. curl -X POST -u Administrator:asdasd http://localhost:9102/settings --data '{"indexer.settings.persisted_snapshot.moi.interval":60000}' 5. Do more mutations for a few minutes to create a couple of disk snapshots. 6. While there is incoming data load in memcached, kill memcached on one node and failover the node in quick succession. 7. Failover shouldn't cause indexes to rollback to 0 and then rebuild again. Also, it would be good to run all rollback related tests to make sure there is no regression.

            People

              mihir.kamdar Mihir Kamdar (Inactive)
              deepkaran.salooja Deepkaran Salooja
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty