Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
5.5.0
-
Component test cluster
172.23.104.16
172.23.104.17
172.23.104.18
172.23.104.19
172.23.104.21
-
Untriaged
-
Centos 64-bit
-
-
Unknown
Description
Enterprise Edition 5.5.0 build 2096
We are running following steps in loop
- Create 4 node cluster 1KV,1 Eventing, 1 indexer ,1 Query
- Start populating data (Note : Mutations are running during full test)
- Create and deploy 2 handlers bucket op and cron timer
- Add one eventing and rebalance
- Undeploy handlers
- Wait for 5 min
- Deploy the same handlers again
- Remove eventing node which is created as a initial setup
- Rebalance hangs at 60%
seeing the following error in eventing logs
2018-03-09T02:39:26.598-08:00 [Error] Consumer::vbTakeoverCallback [worker_cron_timer_op_2:/tmp/127.0.0.1:8091_worker_cron_timer_op_2.sock:5333] vb: 851 vbTakeover request, err: vbucket is owned by another worker on same node |
2018-03-09T02:39:26.599-08:00 [Error] Consumer::vbTakeoverCallback [worker_cron_timer_op_2:/tmp/127.0.0.1:8091_worker_cron_timer_op_2.sock:5333] vb: 853 vbTakeover request, err: vbucket is owned by another worker on same node |
2018-03-09T02:39:26.623-08:00 [Error] Consumer::getOpCallback [worker_cron_timer_op_0:/tmp/127.0.0.1:8091_worker_cron_timer_op_0.sock:5332] Bucket fetch failed for key: <ud>cron_timer_op::vb::172</ud>, err: network error |
2018-03-09T02:39:26.624-08:00 [Error] Consumer::getOpCallback [worker_cron_timer_op_0:/tmp/127.0.0.1:8091_worker_cron_timer_op_0.sock:5332] Bucket fetch failed for key: <ud>cron_timer_op::vb::514</ud>, err: network error |
2018-03-09T02:39:26.624-08:00 [Error] Consumer::getOpCallback [worker_cron_timer_op_0:/tmp/127.0.0.1:8091_worker_cron_timer_op_0.sock:5332] Bucket fetch failed for key: <ud>cron_timer_op::cron_timer::vb::538</ud>, err: network error |
2018-03-09T02:39:26.624-08:00 [Error] Consumer::getOpCallback [worker_cron_timer_op_0:/tmp/127.0.0.1:8091_worker_cron_timer_op_0.sock:5332] Bucket fetch failed for key: <ud>cron_timer_op::vb::649</ud>, err: network error |
Attachments
| For Gerrit Dashboard: MB-28603 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 90908,1 | [MB-28603] Exit the retry loop if network error is encountered | unstable | eventing | Status: ABANDONED | +1 | 0 |
| 91155,1 | [MB-28603] Leverage CAS when updating metadata during rebalance | unstable | eventing | Status: ABANDONED | 0 | 0 |
| 91157,2 | [MB-28603] Leverage node UUIDs as well to check if Eventing node is... | unstable | eventing | Status: MERGED | +2 | +1 |
| 91161,2 | [MB-28603] Perform data transfer related to doc timer before updating... | unstable | eventing | Status: MERGED | +2 | +1 |