Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4344

dispatcher stops persisting items to disk after a successful rebalance due to a race condition in the scheduling ( happens when rebalncing in more than 1 node)

    Details

      Description

      This behavior have been seen while adding 2 or more nodes. steps to reproduce is
      1- create a cluster of N nodes
      2- add two nodes and rebalance
      3- dispatcher does not seem to pick up the items from checkpoints and persist them to disk

      because of this behavior we will have to keep all the closed checkpoints in memory and the node might run out of memory after a while depending on the number of mutations that happen in the cluster.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        farshid Farshid Ghods (Inactive) created issue -
        Show
        farshid Farshid Ghods (Inactive) added a comment - https://github.com/membase/ep-engine/commit/9a21b04ba1863a855acd61243ad26da2d6879c01 https://github.com/membase/ep-engine/commit/9a21b04ba1863a855acd61243ad26da2d6879c01
        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        due to this bug membase server will not be able to persist open checkpoints and it will keep them in the memory for a longer time and depending on the number of mutations the client might see temp OOM errors in that node.

        Show
        farshid Farshid Ghods (Inactive) added a comment - due to this bug membase server will not be able to persist open checkpoints and it will keep them in the memory for a longer time and depending on the number of mutations the client might see temp OOM errors in that node.
        farshid Farshid Ghods (Inactive) made changes -
        Field Original Value New Value
        Assignee Chiyoung Seo [ chiyoung ]
        Fix Version/s 1.7.2 [ 10203 ]
        Show
        farshid Farshid Ghods (Inactive) added a comment - https://github.com/membase/ep-engine/commit/9a21b04ba1863a855acd61243ad26da2d6879c01 https://github.com/membase/ep-engine/commit/9a21b04ba1863a855acd61243ad26da2d6879c01
        farshid Farshid Ghods (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        farshid Farshid Ghods (Inactive) made changes -
        Summary dispatcher stops persisting items after a successful rebalance due to a race condition in the scheduling ( happens when rebalncing in more than 1 node) dispatcher stops persisting items to disk after a successful rebalance due to a race condition in the scheduling ( happens when rebalncing in more than 1 node)
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.7.1-release-notes 1.7.1-release-notes 1.7.2-release-notes
        farshid Farshid Ghods (Inactive) made changes -
        Component/s couchbase-bucket [ 10173 ]
        Component/s ep_engine [ 10013 ]
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            chiyoung Chiyoung Seo
            Reporter:
            farshid Farshid Ghods (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes