Description
Client reported an issue due to a unsatisfactory performance obtained in multigets, that is the case in which In order to obtain a response the service needs multiple gets from the couchbase db
IN PS we have conducted a benchmarking analysis trying to understand whats the best configuration for performance for this particular use case
For our analysis we have benchmarked results (ops/sec and ms per operation) considering 4 groups of different code combinations , multiple sizes of multiget and multiple sizes of the document.
The groups are as follows,
2 groups that depend on whether or not a Jackson serialiser/deserialiser is used to obtain an object class in the form of a POJO
getNoMapper | getWithMapper
2 groups that depend on whether or not all the data is brought back or only specific fields.
short | AllFields
Therefore the 4 code configurations are
getNoMapperShort
getNoMapperAllFields
getWithMapperShort and
getWithMapperAllFields
For each one of the groups there would be entries for all combinations of batch (multiget ) size and document size.
For batch sizes we have considered 10 ,100, 500, 1000
In term of document sizes we have considered the following sizes: 500B 1K 8K18K 50K 100K 200K. The java code used to obtain the results is also attached.
This will be represented in results.csv file as
<code representation> <batch><size><other fields> <throughput or ms per op>
Example
getNoMapperShort 500,50,4000,25000,,404.349, avgt,3, ,151.864,ms/o
Meaning We use no mapper and bring only certain columns of the data on batch size 500 and 50k document size takes 151.864 ms
In our analysis we have determined that the size factor( size doc * batch size ) and ms/op are linearly correlated with R better than 0.98. The Response time is linearly related with the size factor. Also, worth mentioned that the correlation of size factor and throughput is 0.8-0.9
This task is to evaluate what other factors can be considered to boost performance in multigets since this is at sensitive topic at this time for the client.
Attached all the information needed
Java project
Documents of all size
results