Details
-
Bug
-
Resolution: Fixed
-
Test Blocker
-
5.5.0
-
Untriaged
-
-
No
Description
Build : 5.5.0-2542
The longevity system test has a step to add a new indexer node to the cluster and rebalance. Rebalance fails at this step with the following error:
Rebalance exited with reason {service_rebalance_failed,index,
|
{linked_process_died,<21433.11637.7>,
|
{timeout,
|
{gen_server,call,
|
[<21433.8347.0>,
|
{call,"ServiceAPI.GetCurrentTopology",
|
#Fun<json_rpc_connection.0.125340786>},
|
60000]}}}}
|
Error logs on 172.23.99.20 says :
[ns_server:error,2018-04-19T03:16:15.029-07:00,ns_1@172.23.99.20:service_agent-index<0.7761.0>:service_agent:handle_info:231]Linked process <0.11637.7> died with reason {timeout,
|
{gen_server,call,
|
[<0.8347.0>,
|
{call,
|
"ServiceAPI.GetCurrentTopology",
|
#Fun<json_rpc_connection.0.125340786>},
|
60000]}}. Terminating
|
[ns_server:error,2018-04-19T03:16:15.029-07:00,ns_1@172.23.99.20:service_agent-index<0.7761.0>:service_agent:terminate:260]Terminating abnormally
|
[ns_server:error,2018-04-19T03:16:15.029-07:00,ns_1@172.23.99.20:service_agent-index<0.7761.0>:service_agent:terminate:265]Terminating json rpc connection for index: <0.8347.0>
|
[ns_server:error,2018-04-19T03:16:15.037-07:00,ns_1@172.23.99.20:service_agent-index<0.11426.12>:service_agent:handle_call:182]Got rebalance-only call {if_rebalance,<20779.23599.50>,unset_rebalancer} that doesn't match rebalancer pid undefined
|
Also the indexer logs are filled with errors like the following. Not sure if that is related :
2018-04-19T03:34:14.162-07:00 [Info] SCAN##92302 REQUEST defnId:12905143353521314278, instId:15437697085705402619, index:default/default_result_ratings_claims_pindex, type:scan, partitions:[3 5 6 15], scans: [{Low:[null] High:nil Incl:0 ScanType:range Filters:[{CompositeFilters:[{Low:null High:nil Inclusion:0}] Low:[null] High:nil Inclusion:3 ScanType:}] Equals:<nil>}], limit:9223372036854775807, consistency:any_consistency, requestId:94f92668-8337-479f-b820-e8f189b09390, groupaggr: Groups: &{EntryKeyId:0 KeyPos:0 Expr:<nil> ExprValue:<nil>} Aggregates: &{AggrFunc:SUM EntryKeyId:4 KeyPos:1 Expr:<nil> ExprValue:<nil> Distinct:false} DependsOnIndexKeys [0 1] IndexKeyNames [(`default`.`result`) (`default`.`rating`) (`default`.`claim`) (meta(`default`).`id`)] NeedDecode true NeedExplode true IsLeadingGroup true
|
2018-04-19T03:34:14.163-07:00 [Info] SCAN##92302 RESPONSE status:(error = Requested Partial Aggr true Not Supported For Given Scan), requestId: 94f92668-8337-479f-b820-e8f189b09390
|
2018-04-19T03:34:14.164-07:00 [Error] ScanRequest::validateGroupAggr Requested Partial Aggr true Not Supported For Given Scan
|
|
Attachments
For Gerrit Dashboard: MB-29337 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
92990,1 | MB-29337 relax the partial aggregate check | unstable | indexing | Status: MERGED | +2 | +1 |
95084,1 | MB-29337: Disable dynamic writer tuning | unstable | indexing | Status: MERGED | +2 | +1 |