Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4864

rebalancing can get stuck due to a bug detecting the backfill completion during vbucket takeover

    Details

      Description

      Install couchbase server 1.8.0 release with hotfix mb-4738 on 4 nodes (12 GB RAM each) cluster in ec2.
      Load 38 million items to cluster.
      Resident ratio: 41%
      Data size on disk: 78GB
      Remove a node.
      Rebalance OK.
      Add another node in (not the removed node).
      Rebalance hang at around 80+%

      1. 107.22.11.161_tap.txt
        4 kB
        Thuan Nguyen
      2. 107.22.70.136_tap.txt
        20 kB
        Thuan Nguyen
      3. 107.22.84.123_stat.txt
        8 kB
        Thuan Nguyen
      4. 107.22.84.123_tap.txt
        22 kB
        Thuan Nguyen
      5. 107.22.84.123_tap.txt
        18 kB
        Thuan Nguyen
      6. 23.20.45.23_stat.txt
        8 kB
        Thuan Nguyen
      7. 23.20.45.23_tap.txt
        17 kB
        Thuan Nguyen
      8. 23.20.45.23_tap.txt
        16 kB
        Thuan Nguyen
      9. 23.20.50.242_stat.txt
        8 kB
        Thuan Nguyen
      10. 23.20.50.242_tap.txt
        24 kB
        Thuan Nguyen
      11. 23.20.50.242_tap.txt
        20 kB
        Thuan Nguyen
      12. 50.17.157.98_stat.txt
        7 kB
        Thuan Nguyen
      13. 50.17.157.98_tap.txt
        15 kB
        Thuan Nguyen
      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        thuan Thuan Nguyen created issue -
        thuan Thuan Nguyen made changes -
        Field Original Value New Value
        Attachment 23.20.45.23_tap.txt [ 12465 ]
        Attachment 23.20.50.242_tap.txt [ 12466 ]
        Attachment 107.22.11.161_tap.txt [ 12467 ]
        Attachment 107.22.70.136_tap.txt [ 12468 ]
        Attachment 107.22.84.123_tap.txt [ 12469 ]
        thuan Thuan Nguyen made changes -
        Assignee Chiyoung Seo [ chiyoung ]
        chiyoung Chiyoung Seo made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        dipti Dipti Borkar made changes -
        Fix Version/s 1.8.1 [ 10295 ]
        farshid Farshid Ghods (Inactive) made changes -
        Component/s couchbase-bucket [ 10173 ]
        Component/s ep_engine [ 10013 ]
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.8.0-release-notes
        farshid Farshid Ghods (Inactive) made changes -
        Summary rebalance in hang in longevity test cluster rebalancing can get stuck due to a bug detecting the backfill completion during vbucket takeover
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.8.0-release-notes 1.0-release-notes 1.8.0-release-notes
        farshid Farshid Ghods (Inactive) made changes -
        Labels 1.0-release-notes 1.8.0-release-notes 1.8.0-release-notes 1.8.1-release-notes
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        anil Anil Kumar made changes -
        Fix Version/s 2.0-beta [ 10113 ]
        Fix Version/s 1.8.2 [ 10249 ]

          People

          • Assignee:
            chiyoung Chiyoung Seo
            Reporter:
            thuan Thuan Nguyen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes