Details
Description
Install couchbase server 2.0.0-1645 on 11 nodes
Create a 10 nodes cluster.
Create 2 buckets, default and saslbucket.
Load 7 million items to both bucket.
Do rebalance in and out cluster.
I see a lot of repeated error in ns_doctor in log page
[ns_server:debug,2012-08-28T15:09:15.647,ns_1@10.3.121.17:'capi_ddoc_replication_srv-default':cb_generic_replication_srv:handle_info:144]doing replicate_newnodes_docs
[ns_server:info,2012-08-28T15:09:15.638,ns_1@10.3.121.17:ns_log:ns_log:handle_cast:125]suppressing duplicate log menelaus_web:19("Server error during processing: [\"web request failed\",\n
,\n
{type,exit},\n {what,\n {timeout,\n {gen_server,call,[ns_doctor,get_nodes]}}},\n {trace,\n [{gen_server,call,2},\n {ns_doctor,build_tasks_list,2},\n {menelaus_web,handle_tasks,2},\n {menelaus_web,loop,3},\n {mochiweb_http,headers,5},\n {proc_lib,init_p_do_apply,3}]}]") because it's been seen 1 times in the past -5.73e-4 secs (last seen -5.73e-4 secs ago[menelaus:warn,2012-08-28T15:09:15.653,ns_1@10.3.121.17:<0.12668.313>:menelaus_web:loop:419]Server error during processing: ["web request failed",
{path,"/pools/default"},
{type,exit}
,
{what,
{timeout,
{trace,
[{gen_server,call,2},
{menelaus_web,build_pool_info,4},
{menelaus_web,handle_pool_info_wait_tail,5},
{menelaus_web,check_and_handle_pool_info,2},
{menelaus_web,loop,3},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]}]
[menelaus:warn,2012-08-28T15:09:15.816,ns_1@10.3.121.17:<0.31233.312>:menelaus_web:loop:419]Server error during processing: ["web request failed",
{path,"/pools/default"},
{type,exit},
{what,
{timeout,
{gen_server,call, [ns_doctor,get_tasks_version]}
}},
{trace,
[
{menelaus_web,build_pool_info,4},
{menelaus_web,handle_pool_info_wait_tail,5},
{menelaus_web,check_and_handle_pool_info,2},
{menelaus_web,loop,3},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]}]
Server error during processing: ["web request failed", {path,"/pools/default"}, {type,exit},
{what,
{timeout, {gen_server,call, [ns_doctor,get_tasks_version]}}},
{trace,
[{gen_server,call,2}
,
{menelaus_web,build_pool_info,4}, {menelaus_web,handle_pool_info_wait_tail,5}, {menelaus_web,check_and_handle_pool_info,2}, {menelaus_web,loop,3}, {mochiweb_http,headers,5}, {proc_lib,init_p_do_apply,3}]}] (repeated 1 times)Server error during processing: ["web request failed", {path,"/pools/default"}, {type,exit},
{what,
{timeout, {gen_server,call, [ns_doctor,get_tasks_version]}}},
{trace,
[{gen_server,call,2},{menelaus_web,build_pool_info,4}
,
{menelaus_web,handle_pool_info_wait_tail,5},
{menelaus_web,check_and_handle_pool_info,2},
{menelaus_web,loop,3},
{mochiweb_http,headers,5},
{proc_lib,init_p_do_apply,3}]}]
Link to diags of all nodes
https://s3.amazonaws.com/packages.couchbase/diag-logs/large_cluster_2_0/11nodes-1645-ns_doctor-log-page-20120828.tgz