30% increase in autofailover and hardfailover times

http://showfast.sc.couchbase.com/#/timeline/Linux/reb/failover/all

Auto failover (ms), 4 -> 3, 1 bucket x 100M x 2KB, 10K ops/sec

Hard failover (ms), 4 -> 3, 1 bucket x 100M x 2KB, 10K ops/sec

Both tests have have regressed by about 30%. From the logs it looks like each stage of autofailover detection and execution are taking slightly longer. We don't collect stats for these test. Also, I am unable to reach the changelog site, so not sure yet exactly which build is causing the issue. Here is what I have run so far for Autofailover:

5177 - http://perf.jenkins.couchbase.com/job/hestia/6327/ - 213
5189 - http://perf.jenkins.couchbase.com/job/hestia/6334/ - 219
5199 - http://perf.jenkins.couchbase.com/job/hestia/6335/ - 237
5203 - http://perf.jenkins.couchbase.com/job/hestia/6340/ - 243
5204 - http://perf.jenkins.couchbase.com/job/hestia/6341/ - 299
5206 - http://perf.jenkins.couchbase.com/job/hestia/6338/ - 277
5207 - http://perf.jenkins.couchbase.com/job/hestia/6339/ - 323
5209 - http://perf.jenkins.couchbase.com/job/hestia/6336/ - 313
5219 - http://perf.jenkins.couchbase.com/job/hestia/6330/ - 296