Details
-
Bug
-
Resolution: User Error
-
Major
-
5.5.0
-
Untriaged
-
-
Yes
Description
Build : 5.5.0-2451
Saw this issue occur once at the step where indexer nodes are swap rebalanced. Rebalance failed with the following error in the diag log:
Rebalance exited with reason {service_rebalance_failed,index,
{linked_process_died,<23297.7840.1>,
{timeout,
{gen_server,call,
[<23297.20876.0>,
,
60000]}}}}
error.log on index node 172.23.106.96 had some errors like :
[ns_server:error,2018-04-09T19:00:58.866-07:00,ns_1@172.23.106.96:service_agent-index<0.20772.0>:service_agent:handle_info:231]Linked process <0.7840.1> died with reason {timeout,
|
{gen_server,call,
|
[<0.20876.0>,
|
{call,"ServiceAPI.GetTaskList",
|
#Fun<json_rpc_connection.0.125340786>},
|
60000]}}. Terminating
|
[ns_server:error,2018-04-09T19:00:58.867-07:00,ns_1@172.23.106.96:service_agent-index<0.20772.0>:service_agent:terminate:260]Terminating abnormally
|
[ns_server:error,2018-04-09T19:00:58.867-07:00,ns_1@172.23.106.96:service_agent-index<0.20772.0>:service_agent:terminate:265]Terminating json rpc connection for index: <0.20876.0>
|
[ns_server:error,2018-04-09T19:00:58.868-07:00,ns_1@172.23.106.96:service_agent-index<0.19037.1>:service_agent:handle_call:182]Got rebalance-only call {if_rebalance,<21161.8716.5>,unset_rebalancer} that doesn't match rebalancer pid undefined
|
[ns_server:error,2018-04-09T19:01:04.311-07:00,ns_1@172.23.106.96:ns_heart_slow_status_updater<0.20150.0>:ns_heart:grab_one_service_status:498]Failed to grab service index status: {exit,
|
{timeout,
|
{gen_server,call,
|
['service_agent-index',get_status,
|
2000]}},
|
[{gen_server,call,3,
|
[{file,"gen_server.erl"},{line,188}]},
|
{ns_heart,grab_one_service_status,1,
|
[{file,"src/ns_heart.erl"},
|
{line,495}]},
|
{ns_heart,
|
'-grab_service_statuses/0-lc$^1/1-1-',
|
1,
|
[{file,"src/ns_heart.erl"},
|
{line,491}]},
|
{ns_heart,current_status_slow_inner,0,
|
[{file,"src/ns_heart.erl"},
|
{line,354}]},
|
{ns_heart,current_status_slow,1,
|
[{file,"src/ns_heart.erl"},
|
{line,250}]},
|
{ns_heart,slow_updater_loop,0,
|
[{file,"src/ns_heart.erl"},
|
{line,244}]}]}
|
|