Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7519

Has better batch scheduling mechanism for Couchstore I/O

    Details

      Description

      With MRW development (Multiple Reader Writer) EP Engine can further optimize current batching mechanism that reduces number of interim queus for pending SETs and exercises better scheduling policy for each SET.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-ep-engine-2-0 #477 (See http://qa.hq.northscale.net/job/github-ep-engine-2-0/477/)
        MB-7519: Better flusher batching for writes (Revision f8d135bee69b61086ab00850f6a72630bac3e34b)

        Result = SUCCESS
        Mike Wiederhold :
        Files :

        • src/ep_engine.cc
        • src/tapthrottle.cc
        • management/cbepctl
        • src/checkpoint.hh
        • src/checkpoint.cc
        • src/ep.cc
        • src/flusher.cc
        • docs/stats.org
        • src/ep_engine.h
        • src/flusher.hh
        • tests/module_tests/timing_tests.cc
        • src/item_pager.cc
        • src/mutation_log_compactor.cc
        • src/vbucket.cc
        • src/vbucket.hh
        • src/ep.hh
        • src/stats.hh
        • tests/ep_test_apis.cc
        Show
        thuan Thuan Nguyen added a comment - Integrated in github-ep-engine-2-0 #477 (See http://qa.hq.northscale.net/job/github-ep-engine-2-0/477/ ) MB-7519 : Better flusher batching for writes (Revision f8d135bee69b61086ab00850f6a72630bac3e34b) Result = SUCCESS Mike Wiederhold : Files : src/ep_engine.cc src/tapthrottle.cc management/cbepctl src/checkpoint.hh src/checkpoint.cc src/ep.cc src/flusher.cc docs/stats.org src/ep_engine.h src/flusher.hh tests/module_tests/timing_tests.cc src/item_pager.cc src/mutation_log_compactor.cc src/vbucket.cc src/vbucket.hh src/ep.hh src/stats.hh tests/ep_test_apis.cc
        Hide
        maria Maria McDuff (Inactive) added a comment -

        abhinav,

        this is one of the features in 2.0.2 where small load litmus will be run by QE (abhinav).

        IO Batching improvements Mike Abhinav (small load) Ronnie / Pavel Do litmus

        https://docs.google.com/spreadsheet/ccc?key=0AqedrMvnwW3FdHJkQ3h3WS1SRVF1a1VkU2NjMG1ZX0E#gid=1

        Show
        maria Maria McDuff (Inactive) added a comment - abhinav, this is one of the features in 2.0.2 where small load litmus will be run by QE (abhinav). IO Batching improvements Mike Abhinav (small load) Ronnie / Pavel Do litmus https://docs.google.com/spreadsheet/ccc?key=0AqedrMvnwW3FdHJkQ3h3WS1SRVF1a1VkU2NjMG1ZX0E#gid=1
        Hide
        maria Maria McDuff (Inactive) added a comment -

        per mike:

        I spent a lot of time verifying this fix and making sure that we do have improved batch scheduling. If you want to test this I recommend only doing performance testing. I already did some of this and found a very small improvement, but the big win here in my mind is simplification of code. I would only reopen this issue if we saw a performance regression.

        Show
        maria Maria McDuff (Inactive) added a comment - per mike: I spent a lot of time verifying this fix and making sure that we do have improved batch scheduling. If you want to test this I recommend only doing performance testing. I already did some of this and found a very small improvement, but the big win here in my mind is simplification of code. I would only reopen this issue if we saw a performance regression.
        Hide
        ketaki Ketaki Gangal added a comment -

        Some preliminary perf tests show improvement in Drain rate
        http://www.couchbase.com/issues/browse/CBD-918

        Show
        ketaki Ketaki Gangal added a comment - Some preliminary perf tests show improvement in Drain rate http://www.couchbase.com/issues/browse/CBD-918
        Hide
        maria Maria McDuff (Inactive) added a comment - - edited

        Thanks Abhinav, Ketaki.

        Hi Abhinav — Once we get the build with MWR and the 2.0.2 feature set, pls re-run the same tests. Thanks.

        https://docs.google.com/spreadsheet/ccc?key=0AqedrMvnwW3FdHJkQ3h3WS1SRVF1a1VkU2NjMG1ZX0E#gid=0

        Show
        maria Maria McDuff (Inactive) added a comment - - edited Thanks Abhinav, Ketaki. Hi Abhinav — Once we get the build with MWR and the 2.0.2 feature set, pls re-run the same tests. Thanks. https://docs.google.com/spreadsheet/ccc?key=0AqedrMvnwW3FdHJkQ3h3WS1SRVF1a1VkU2NjMG1ZX0E#gid=0
        Hide
        kzeller kzeller added a comment -

        1) For RN and DOC or just RN?
        2) I think this means that MRW implementation also included i/O performance for individual threads. Confirm? If incorrect, please explain what this improvement is?

        Show
        kzeller kzeller added a comment - 1) For RN and DOC or just RN? 2) I think this means that MRW implementation also included i/O performance for individual threads. Confirm? If incorrect, please explain what this improvement is?
        Hide
        jin Jin Lim (Inactive) added a comment -

        Per PM (Anil) it sounded like not a RN candidate:
        From Anil's previous email:
        "Okay, so its not RN candidate but we can document this enhancement in 2.1.0 but not in-terms of performance gain."

        This batch scheduling mechanism is independent of MRW. This enhancement mainly for better scheduling of batching Couchstore I/O for each individual thread (reader or writer). Hope this clarifies things better.

        Show
        jin Jin Lim (Inactive) added a comment - Per PM (Anil) it sounded like not a RN candidate: From Anil's previous email: "Okay, so its not RN candidate but we can document this enhancement in 2.1.0 but not in-terms of performance gain." This batch scheduling mechanism is independent of MRW. This enhancement mainly for better scheduling of batching Couchstore I/O for each individual thread (reader or writer). Hope this clarifies things better.
        Hide
        kzeller kzeller added a comment -

        Per QA/ENG = not a release note candidate.

        Show
        kzeller kzeller added a comment - Per QA/ENG = not a release note candidate.

          People

          • Assignee:
            kzeller kzeller
            Reporter:
            jin Jin Lim (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes