Investigate GSIclient <-> Indexer connection implementation for performance bottleneck

In performance tests, we saw concurrent range queries to give less than 2000 rows/sec throughput. This ticket is to investigate GSIclient <-> Indexer connection implementation. Details from email thread below:

+ Pratap

GsiClient has 20 connections which can go in parallel. I have double check with Pratap.

Venu, you can also open a bug and we can follow up the testing from there.

John

From: Sriram Melkote <siri@couchbase.com>
Date: Monday, April 13, 2015 at 12:33 PM
To: Venu Uppalapati <venu@couchbase.com>, Sarath Lakshman <sarath@couchbase.com>, John Liang <john.liang@couchbase.com>
Subject: Re: scan throughput issue

Hi Sarath,

Can you please help with this? Venu said this is many times slower than what we'd expect.

One thing that's not clear to me is the impact of the synchronous GsiClient <-> Indexer protocol.

Will this benefit by using multiple GSIClient instances?

Thanks,
SIri

On Apr 11, 2015, at 3:26 AM, Venu Uppalapati <venu@couchbase.com> wrote:

Hi Siri,

Here are the gerrit review submits for the concurrent scan throughput tests. In commit 2) the callback body is identical to the original cbindexperf implementation while in commit 1) the callback has no body. I see similar throughput in both cases. I have reviewed the code by myself and John took a look as well but could not identify any bottlenecks in the test code. The test was run against indexes of sizes 20M and 50M items. The single range span is 24 rows, config file is attached. Please let me know if any additional information is needed.

1)http://review.couchbase.org/#/c/49403/
2)http://review.couchbase.org/#/c/49525/

Thanks,
Venu
<config_rangethr.json>