Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.6.0
-
7.6.0-1766
-
Untriaged
-
0
-
Unknown
Description
Rebalance failure with this reason -
Rebalance exited with reason {service_rebalance_failed,index,
|
{worker_died,
|
{'EXIT',<0.6998.147>, |
{task_failed,rebalance,
|
{service_error,
|
<<"shard copy aborted: LSS data copy failed :plasma log copy failed, dst path :https://172.23.106.171:9104//plasma_storage_v1/442dbce4c37f9c821c90070211655e48_ShardToken99_49_23_67_18_d4_69_47/5074468002420624045/shards/shard5074468002420624045/data/recovery error :log segment copy failed, src:/data/@2i/shards/shard5074468002420624045/data/recovery/log.00000000000000.data dst:https://172.23.106.171:9104//plasma_storage_v1/442dbce4c37f9c821c90070211655e48_ShardToken99_49_23_67_18_d4_69_47/5074468002420624045/shards/shard5074468002420624045/data/recovery/log.00000000000000.data off:24665386, written: 17408726 error: FileTransfer error, dstPath :https://172.23.106.171:9104//plasma_storage_v1/442dbce4c37f9c821c90070211655e48_ShardToken99_49_23_67_18_d4_69_47/5074468002420624045/shards/shard5074468002420624045/data/recovery/log.00000000000000.data error :Post \"https://172.23.106.171:9104/rpc/CloseFile\": context deadline exceeded, shardId :5074468002420624045">>}}}}}. |
Rebalance Operation Id = e6fe2b717a1dffa11083eec3acc743b8
|
Cbcollect logs:
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.105.122.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.106.171.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.106.176.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.106.30.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.96.198.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.96.230.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.96.245.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.97.100.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.97.108.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.97.109.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.97.66.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1699463374/collectinfo-2023-11-08T171830-ns_1%40172.23.97.67.zip
Attachments
For Gerrit Dashboard: MB-59581 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
199918,8 | MB-59581: Add rpc SyncCloseFile | unstable | plasma | Status: MERGED | +2 | +1 |
201214,7 | MB-59581: Increase rpc poll timeout | unstable | plasma | Status: MERGED | +2 | +1 |