cluster stuck post rebalance in plasma longevity
Description
Components
Affects versions
Fix versions
Labels
Environment
Link to Log File, atop/blg, CBCollectInfo, Core dump
Release Notes Description
Attachments
- 01 Mar 2017, 06:13 AM
- 01 Mar 2017, 06:12 AM
- 01 Mar 2017, 06:10 AM
relates to
Activity
Arunkumar Senthilnathan March 14, 2017 at 6:04 AM
Issue not found in build post merge - closing it out
CB robot March 2, 2017 at 4:21 PM
Build 5.0.0-2171 contains nitro commit 6a646995e3a94b9c06d9c4a839ac85179046a387 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-23082#icft=MB-23082 plasma: Handle race conditions during page merge/remove
https://github.com/couchbase/nitro/commit/6a646995e3a94b9c06d9c4a839ac85179046a387
Sarath Lakshman March 2, 2017 at 4:15 PM
Merged fixes to master branch:
http://review.couchbase.org/#/c/74524/
http://review.couchbase.org/#/c/74525/
Arunkumar Senthilnathan March 2, 2017 at 6:28 AM
Cluster was not stuck due to this issue - it was due to crash tracked in https://couchbasecloud.atlassian.net/browse/MB-23106#icft=MB-23106
Arunkumar Senthilnathan March 2, 2017 at 4:41 AM
Same issue observed in ubuntu longevity as well:
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.105.60.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.105.61.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.105.62.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.105.63.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.105.83.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.106.14.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.106.213.zip
https://s3.amazonaws.com/bugdb/jira/plasma/collectinfo-2017-03-02T043146-ns_1%40172.23.106.96.zip
We are running plasma longevity with 200k items in burrito cluster - it got stuck after rebalance since yest evening - indexer crashes also found in logs - attaching logs