write/observe/query latencies are high

woq (write/obs/query) is a standard pattern to understand system latencies.

based on a regular 2 node mixed load perf test, 90th percentile woq latencies are really high, (1000 seconds)

please see the attached graphs.

graph # 1: sanity test,90th percentile latencies are around 45 sec.

graph # 2: mixed 2 node test, 90th percentile latencies are around 1000 sec...

From Steve Yen:

(g)woq means yet-another client thread is doing a workload of the following...

• Get an item
• Write (do a mutation on that item)
• Observe that it persists
• Query some view

And measure latency (Observe and Query time).