[system test] Hard Fail Over -> add back with Full Recovery: Rebalance exited with reason {buckets_shutdown_wait_failed, {old_buckets_shutdown_wait_failed,

steps:
1. 3 nodes in cluster, 4 buckets. run data loader more then a day
2. setup replication from SRC to cluster DEST for all buckets.
3. rebalance in at SRC cluster
rebalance in at DEST cluster
4. Graceful Fail Over(rebalance) for node on SRC, add back(Delta Recovery) and rebalance - PASS
5) Hard Fail Over for node on SRC, add back(Full Recovery) and rebalance

Rebalance exited with reason {buckets_shutdown_wait_failed,
[{'ns_1@172.23.105.166',
{'EXIT',

{old_buckets_shutdown_wait_failed, ["MsgsCalls","RevAB"]}

}}]}
ns_orchestrator002 ns_1@172.23.105.163 08:17:59 - Thu Jun 18, 2015
Failed to wait deletion of some buckets on some nodes: [{'ns_1@172.23.105.166',
{'EXIT',

{old_buckets_shutdown_wait_failed, ["MsgsCalls", "RevAB"]}

}}]
ns_rebalancer000 ns_1@172.23.105.163 08:17:59 - Thu Jun 18, 2015
Service 'projector' exited with status 1. Restarting. Messages: github.com/couchbase/indexing/secondary/common.ExitOnStdinClose()
C:/Jenkins/workspace/sherlock-windows/couchbase/goproj/src/github.com/couchbase/indexing/secondary/common/util.go:308 +0xa0
created by main.main
C:/Jenkins/workspace/sherlock-windows/couchbase/goproj/src/github.com/couchbase/indexing/secondary/cmd/projector/main.go:93 +0x916
[goport] 2015/06/18 08:17:51 c:/Program Files/Couchbase/Server/bin/projector.exe terminated: exit status 2 ns_log000 ns_1@172.23.105.166 08:17:56 - Thu Jun 18, 2015
Starting rebalance, KeepNodes = ['ns_1@172.23.105.163','ns_1@172.23.105.164',
'ns_1@172.23.105.165','ns_1@172.23.105.166'], EjectNodes = [], Failed over and being ejected nodes = []; no delta recovery nodes
ns_orchestrator004 ns_1@172.23.105.163 08:17:38 - Thu Jun 18, 2015