- Create a 6 nodes cluster: 3KV, 3 N1ql/GSI
- Create a bucket, 2 collections, load 1.5b items in each.
- Build GSI indexes. Wait for them to be ready
- Start a query load asynchronously. 200 threads running queries in a loop. In each iteration query is chosen randomly from 5 queries.
- Start KV r:w mutations at 10K/s, 50-50%
- Change the disk which results in swap rebalance.
- 3 KV nodes swap rebalance finished in 7.5 hours
- 3 GSI node swap rebalance finished in 28.5 hours
- During the last GSI/N1QL node swap rebalance max number of queries are timing out.
- In UI, I don't see any data available in stats between 10pm May 16th to 6am - May 17th. That was the time i saw max queries timing out.
cc: Dave Rigby