Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37680

Order DCP streams when streaming from multiple VB

    XMLWordPrintable

Details

    Description

      Problem

      For consistency cbbackupmgr is forced to open all vbucket streams at once.  This a has a performance impact because the streams are interlaced, which causes cbbackupmgr to jump between different vbucket files.

      For supporting backup to S3 it is too expensive to hold all 1024 vbuckets open at once, because to be cost efficient the upload chunks have to be large.

      A possible solution is to open and stream one vbucket at a time, however this causes consistency problems.

      It is suggested that a DCP control message cold be used to state that it will open the file handle to all vbuckets, which gives consistency, however stream only one vbucket at a time

       

       

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-37680
          # Subject Branch Project Status CR V

          Activity

            drigby Dave Rigby added a comment -

            Proposed solution - allow a DCP client to request sequential backfill scheduling instead of the default round-robin via a new DCP control message:

            • stream_backfill_order Request the order that backfilling streams should be scheduled. Permitted values:
              • round-robin (Default) Schedule backfilling of streams in round-robin order, attempting to (appromixately) send data of similar ages at a time. Default (and only) backfill order before this change.
              • sequential Schedule backfilling streams in sequential order, backfilling all items from the first stream before advancing to the second, etc.
            drigby Dave Rigby added a comment - Proposed solution - allow a DCP client to request sequential backfill scheduling instead of the default round-robin via a new DCP control message: stream_backfill_order Request the order that backfilling streams should be scheduled. Permitted values: round-robin (Default) Schedule backfilling of streams in round-robin order, attempting to (appromixately) send data of similar ages at a time. Default (and only) backfill order before this change. sequential Schedule backfilling streams in sequential order, backfilling all items from the first stream before advancing to the second, etc.
            owend Daniel Owen added a comment - - edited

            Dave Finlay After thinking it through the new control message will not require any cluster compatibility change; worst case if there are nodes that do not support the new control message, it means that the backup client will be aware that all vbuckets cannot be streamed at the same time. Hence I don't believe any ns_server work is required.

            owend Daniel Owen added a comment - - edited Dave Finlay After thinking it through the new control message will not require any cluster compatibility change; worst case if there are nodes that do not support the new control message, it means that the backup client will be aware that all vbuckets cannot be streamed at the same time. Hence I don't believe any ns_server work is required.

            Build couchbase-server-6.6.0-7640 contains kv_engine commit deb0b00 with commit message:
            MB-37680: Enable Backfill tests for Ephemeral

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7640 contains kv_engine commit deb0b00 with commit message: MB-37680 : Enable Backfill tests for Ephemeral

            Build couchbase-server-6.6.0-7665 contains kv_engine commit 82c0c2a with commit message:
            MB-37680: Simplify Backfill finishing logic

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7665 contains kv_engine commit 82c0c2a with commit message: MB-37680 : Simplify Backfill finishing logic

            Build couchbase-server-6.6.0-7666 contains kv_engine commit 8edbaed with commit message:
            MB-37680: BackfillManager::backfill() - Simplify snooze logic

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7666 contains kv_engine commit 8edbaed with commit message: MB-37680 : BackfillManager::backfill() - Simplify snooze logic

            Build couchbase-server-6.6.0-7667 contains kv_engine commit 57bcba2 with commit message:
            MB-37680: Introduce DCPBackfillIface

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7667 contains kv_engine commit 57bcba2 with commit message: MB-37680 : Introduce DCPBackfillIface

            Build couchbase-server-6.6.0-7668 contains kv_engine commit d63201a with commit message:
            MB-37680: MockActiveStream: default common arguments

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7668 contains kv_engine commit d63201a with commit message: MB-37680 : MockActiveStream: default common arguments

            Build couchbase-server-7.0.0-2124 contains kv_engine commit deb0b00 with commit message:
            MB-37680: Enable Backfill tests for Ephemeral

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2124 contains kv_engine commit deb0b00 with commit message: MB-37680 : Enable Backfill tests for Ephemeral

            Build couchbase-server-6.6.0-7673 contains kv_engine commit 14e4e15 with commit message:
            MB-37680: Add support for sequential DCP Backfill

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7673 contains kv_engine commit 14e4e15 with commit message: MB-37680 : Add support for sequential DCP Backfill

            Build couchbase-server-7.0.0-2222 contains kv_engine commit 82c0c2a with commit message:
            MB-37680: Simplify Backfill finishing logic

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2222 contains kv_engine commit 82c0c2a with commit message: MB-37680 : Simplify Backfill finishing logic

            Build couchbase-server-6.6.0-7731 contains kv_engine commit 9b06cd8 with commit message:
            MB-37680: Reduce differences between eph and persistent backfill

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7731 contains kv_engine commit 9b06cd8 with commit message: MB-37680 : Reduce differences between eph and persistent backfill

            Build couchbase-server-7.0.0-2235 contains kv_engine commit 8edbaed with commit message:
            MB-37680: BackfillManager::backfill() - Simplify snooze logic

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2235 contains kv_engine commit 8edbaed with commit message: MB-37680 : BackfillManager::backfill() - Simplify snooze logic

            Build couchbase-server-7.0.0-2253 contains kv_engine commit 14e4e15 with commit message:
            MB-37680: Add support for sequential DCP Backfill

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2253 contains kv_engine commit 14e4e15 with commit message: MB-37680 : Add support for sequential DCP Backfill

            Build couchbase-server-7.0.0-2253 contains kv_engine commit d63201a with commit message:
            MB-37680: MockActiveStream: default common arguments

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2253 contains kv_engine commit d63201a with commit message: MB-37680 : MockActiveStream: default common arguments

            Build couchbase-server-7.0.0-2253 contains kv_engine commit 57bcba2 with commit message:
            MB-37680: Introduce DCPBackfillIface

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2253 contains kv_engine commit 57bcba2 with commit message: MB-37680 : Introduce DCPBackfillIface

            Build couchbase-server-6.6.0-7766 contains kv_engine commit a31e868 with commit message:
            MB-37680: Schedule backfill of snap_markers ASAP

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7766 contains kv_engine commit a31e868 with commit message: MB-37680 : Schedule backfill of snap_markers ASAP

            Build couchbase-server-7.0.0-2279 contains kv_engine commit 9b06cd8 with commit message:
            MB-37680: Reduce differences between eph and persistent backfill

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2279 contains kv_engine commit 9b06cd8 with commit message: MB-37680 : Reduce differences between eph and persistent backfill

            Build couchbase-server-7.0.0-2321 contains kv_engine commit a31e868 with commit message:
            MB-37680: Schedule backfill of snap_markers ASAP

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2321 contains kv_engine commit a31e868 with commit message: MB-37680 : Schedule backfill of snap_markers ASAP

            People

              owend Daniel Owen
              owend Daniel Owen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty