Description
We have a test that uses few cluster_run nodes. The first node has the services (data,fts,index,query) and the second node has the services (analytics,fts,query). Couple of days ago the test started intermittently failing when attempting to rebalance in the second node with:
[ns_server:error,2019-10-05T02:06:57.620-07:00,n_0@127.0.0.1:<0.1676.0>:service_janitor:maybe_init_topology_aware_service:87]Initial rebalance for `index` failed: {error, |
{initial_rebalance_failed,index,
|
{agent_died,<0.1320.0>, |
{linked_process_died,<0.1321.0>, |
{no_connection,
|
"index-service_api"}}}}} |
The indexing service logs show the following:
2019-10-05T02:06:55.227-07:00 [Error] EmbeddedServer.boostrap: error : FDB_RESULT_TOO_BIG_BUFFER_CACHE |
2019-10-05T02:06:55.227-07:00 [Error] ClustMgrAgent::NewClustMgrAgent Error In Init FDB_RESULT_TOO_BIG_BUFFER_CACHE |
2019-10-05T02:06:55.227-07:00 [Fatal] Indexer::NewIndexer ClusterMgrAgent Init Error FDB_RESULT_TOO_BIG_BUFFER_CACHE |
2019-10-05T02:06:55.227-07:00 [Warn] Indexer Failure to Init FDB_RESULT_TOO_BIG_BUFFER_CACHE |
2019-10-05T02:06:55.227-07:00 [Info] Indexer exiting normally |
Logs from all cluster_run nodes are attached. Please let me know if more information is needed.
Attachments
For Gerrit Dashboard: MB-36342 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
116189,2 | MB-36342: Intermittent Rebalance Failure | cb-master | forestdb | Status: MERGED | +2 | +1 |