Query results in u'msg': u'read tcp <index node>:9101: i/o timeout', u'code': 5000 After failover (index node) +rebalance+index creation

INI
[global]
port:8091
username:root
password:couchbase
index_port=9102
n1ql_port:8903

[servers]
1:SERVER_1
2:SERVER_2
3:SERVER_3
4:SERVER_4
5:SERVER_5
6:SERVER_6

[SERVER_1]
ip:172.23.121.66
services=n1ql,kv

[SERVER_2]
ip:172.23.121.67
services=kv

[SERVER_3]
ip:172.23.121.68
services=kv

[SERVER_4]
ip:172.23.121.69
services=index

[SERVER_5]
ip:172.23.121.70
services=index,kv

[SERVER_6]
ip:172.23.121.71
services=index

[membase]
rest_username:Administrator
rest_password:password

[tuq_client]
goroot:/root/n1ql/go
sherlock_path=/opt/couchbase/bin

testrunner -i /tmp/tuqvm.ini doc-per-day=10,skip_build_tuq=True,cbq_version=sherlock,skip_cleanup=True -t 2i.recovery_2i.SecondaryIndexingRecoveryTests.test_autofailover,nodes_init=5,nodes_out=1,before=,in_between=create_index,after=query_ops,groups=simple,dataset=default,doc-per-day=10,services_init=n1ql:kv-kv-kv-index-index,nodes_out_dist=index:1,GROUP=AUTOFAILOVER;P1

Scenario
1. Create 5 node cluster (1: n1ql:kv 2:kv 3:kv 4:index 5:index)
2. Create default bucket with 20160 items - Employee dataset
3. Create Primary index using views - Wait till it comes online
4. Stop index node (node 4) and rebalance it out.
5. Create GSI Indexes as follows
CREATE INDEX employee23ada1529b6140d3973e741dc19464d8job_title ON default(job_title) USING GSI
CREATE INDEX employee23ada1529b6140d3973e741dc19464d8join_yr ON default(join_yr) USING GSI
6. Wait till the indexes come online
7. Run queries to use the index.
SELECT * FROM default WHERE join_yr > 1999

Here is the explain for the queries

[{u'index': u'employee23ada1529b6140d3973e741dc19464d8join_yr', u'namespace': u'default', u'keyspace': u'default', u'#operator': u'IndexScan', u'limit': 9.223372036854776e+18, u'spans': [{u'Range':

{u'High': None, u'Low': [u'1999'], u'Inclusion': 0}

, u'Seek': None}], u'using': u'gsi'}, {u'#operator': u'Parallel', u'~child': {u'#operator': u'Sequence', u'~children': [

{u'keyspace': u'default', u'#operator': u'Fetch', u'namespace': u'default'}

,

{u'#operator': u'Filter', u'condition': u'(1999 < (`default`.`join_yr`))'}

, {u'#operator': u'InitialProject', u'result_terms': [

{u'star': True}

]},

{u'#operator': u'FinalProject'}

]}}]}

Exception as seen

N1QLQueryException: unable to get expected results for query CBQError: host 172.23.121.66: ERROR:{u'status': u'errors', u'errors': [

{u'msg': u'read tcp 172.23.121.70:9101: i/o timeout', u'code': 5000}

,

{u'msg': u'read tcp 172.23.121.70:9101: i/o timeout', u'code': 5000}

], u'results': [{u'default': {u'tasks_points':

{u'task1': 8, u'task2': 1}

, u'name': u'employee-17', u'mutated': 0, u'skills': [u'skill2010', u'skill2011'], u'join_day': 17, u'email': u'17-mail@couchbase.com', u'test_rate': 5.5, u'join_mo': 5, u'join_yr': 2010, u'_id': u'query-testemployee10513.4473782-7', u'VMs': [

{u'RAM': 5, u'os': u'ubuntu', u'name': u'vm_5', u'memory': 5}

,

{u'RAM': 5, u'os': u'windows', u'name': u'vm_6', u'memory': 5}

], u'job_title': u'Sales'}}, {u'default': {u'tasks_points':

{u'task1': 6, u'task2': 1}

, u'name': u'employee-15', u'mutated': 0, u'skills': [u'skill2010', u'skill2011'], u'join_day': 15, u'email': u'15-mail@couchbase.com', u'test_rate': 9.9, u'join_mo': 9, u'join_yr': 2010, u'_id': u'query-testemployee11871.7634615-5', u'VMs': [

{u'RAM': 9, u'os': u'ubuntu', u'name': u'vm_9', u'memory': 9}

,

{u'RAM': 9, u'os': u'windows', u'name': u'vm_10', u'memory': 9}

], u'job_title': u'Engineer'}}, {u'default': {u'tasks_points':

{u'task1': 9, u'task2': 1}

, u'name': u'employee-18', u'mutated': 0, u'skills': [u'skill2010', u'skill2011'], u'join