[system test] compaction_daemon crashed due to {not_found, no_db_file}

Cluster information:

• 8 centos 6.2 64bit server with 4 cores CPU
• Each server has 32 GB RAM and 400 GB SSD disk.
• 24.8 GB RAM for couchbase server at each node
• SSD disk format ext4 on /data
• Each server has its own drive, no disk sharing with other server.
• Cluster has 2 buckets, default (12GB) and saslbucket (12GB) and setup cluster with consistent enable.
• Each bucket has one doc and 2 views for each doc (default d1 and saslbucket d11)

• Create cluster with 6 nodes installed couchbase server 2.0.0-1746

10.6.2.37
10.6.2.38
10.6.2.39
10.6.2.40
10.6.2.42
10.6.2.43

• Load 28 million items to both bucket. Each key has size from 512 bytes to 1500 bytes

Add 2 nodes 10.6.2.44, 10.6.2.45 and remove 2 node 10.6.2.40, 10.6.2.43
Rebalance. Rebalance seems very slow. After 10 hours of running rebalance, I stop rebalance.
Restart rebalance again. Rebalance failed. Filed bug MB-6745

Check diags file of node 45, see compaction_daemon crashed

[ns_server:warn,2012-09-26T13:20:44.939,ns_1@10.6.2.45:<0.7284.88>:compaction_daemon:do_chain_compactors:519]Compactor for view `default/_design/d1/main` (pid [

{type,view},
{important,true},
{name, <<"default/_design/d1/main">>},
{fa,
{#Fun<compaction_daemon.16.36173935>,
[<<"default">>,
<<"_design/d1">>,main,
{config,
{30, 18446744073709551616},
{30, 18446744073709551616},
undefined,false,
{daemon_config,30,
131072}},
true,
{[{type,view}

,

{name, <<"_design/d1">>}

]}]}}]) terminated unexpectedly:

{not_found, no_db_file}

[error_logger:error,2012-09-26T13:20:44.939,ns_1@10.6.2.45:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: compaction_daemon:spawn_view_index_compactor/6-fun-0/0
pid: <0.7285.88>
registered_name: []
exception throw:

{not_found,no_db_file}
in function couch_set_view:open_set_group/2
in call from couch_set_view:get_group_pid/2
in call from couch_set_view:get_group_data_size/2
in call from compaction_daemon:get_group_data_info/3
in call from compaction_daemon:ensure_can_view_compact/3
in call from compaction_daemon:'spawn_view_index_compactor/6-fun-0'/7
ancestors: [<0.7284.88>,compaction_daemon,<0.26393.1>,ns_server_sup,
ns_server_cluster_sup,<0.58.0>]
messages: []
links: [<0.7284.88>]
dictionary: []
trap_exit: false
status: running
heap_size: 377
stack_size: 24
reductions: 769
neighbours:

[user:error,2012-09-26T13:20:44.939,ns_1@10.6.2.45:compaction_daemon:compaction_daemon:handle_info:324]User-triggered compaction of view `default/_design/d1` failed: {not_found, no_db_file}. See logs for detailed reason.
[error_logger:error,2012-09-26T13:20:44.940,ns_1@10.6.2.45:error_logger:ale_error_logger_handler:log_report:72]
=========================CRASH REPORT=========================
crasher:
initial call: compaction_daemon:spawn_view_compactor/5-fun-1/0
pid: <0.7284.88>
registered_name: []
exception exit: {not_found,no_db_file}

in function compaction_daemon:do_chain_compactors/2
ancestors: [compaction_daemon,<0.26393.1>,ns_server_sup,
ns_server_cluster_sup,<0.58.0>]
messages: []
links: [<0.26396.1>]
dictionary: []
trap_exit: true
status: running
heap_size: 1597
stack_size: 24
reductions: 4116
neighbours:

Link to collect info of all nodes https://s3.amazonaws.com/packages.couchbase/collect_info/orange/2_0_0/201209/8nodes-col-info-1746-compaction_daemon-crashed-no_db_file-20120926-145909.tgz