Details
Description
build: 1409
issues was found when test cluster with ~10,000,000 items with rebalance in/out( mcsoda was used for data ops)
approximate steps:
10.3.121.112
10.3.121.113
10.3.121.114
10.3.121.115
10.3.121.116
1.one node 10.3.121.112
2. upload data pytests/performance/mcsoda.py membase://10.3.121.112:8091 max-items=10000000 json=1 doc-gen=0 doc-cache=0 ratio-sets=0.9 vbuckets=1024
3.rebalance in 10.3.121.113, 10.3.121.114,10.3.121.115 during data loading incrementally
4.stop data loading when number items is 11105199
5.rebalance out 10.3.121.115
6. 10% delete , 100% update data pytests/performance/mcsoda.py membase://10.3.121.112:8091 max-creates=10000000 json=1 doc-gen=0 doc-cache=0 ratio-sets=0 ratio-deletes=0.1 vbuckets=1024
7. rebalance in 10.3.121.115, 10.3.121.116 ( #6 is performed)
8.during rebalancing added production view
at one point the orchestrator ceased to display statistics. all other nodes display stats partially
see screenshot
exception in the logs:
[couchdb:info] [2012-07-05 19:11:39] [ns_1@10.3.121.112:<0.5714.19>:couch_log:info:39] Updater checkpointing set view `default` update for main group `_design/dev_AA`
[stats:error] [2012-07-05 19:11:42] [ns_1@10.3.121.112:<0.7243.0>:stats_collector:handle_info:96] Exception in stats collector: {exit,
{timeout,
{gen_server,call,
[
,
fetch_stats]}},
[
,
,
,
,
,
]}