Major Description
I'm working with Intel on MDS performance testing and here's how the cluster set up.
112: Data Service Only.
92: Index/Query service. (MOI, N1QL)
Version: Couchbase Server 4.5 Beta.
I started YCSB workloadE from client and here's are the error messages:
com.couchbase.client.core.RequestCancelledException: Could not dispatch request, cancelling instead of retrying.
at com.couchbase.client.core.retry.RetryHelper.retryOrCancel(RetryHelper.java:45)
at com.couchbase.client.core.node.locate.QueryLocator.locateAndDispatch(QueryLocator.java:48)
at com.couchbase.client.core.RequestHandler.dispatchRequest(RequestHandler.java:219)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:176)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:71)
at com.couchbase.client.deps.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Matt and I looked into use strace and we found out there's SIGSEGV errors in the log.
See attachment.
Full logs are here:
https://drive.google.com/drive/folders/0B1b0nb7ZnKdzM0V1eS1fVkYtVG8
I looked it up online and I see this:
https://github.com/golang/go/issues/10984
This seems to be relate to Go runtime crash.
Intel Env is behind firewall and I have access. Please let me know if you need access.
Command to run from client side:
strace -f -o workloade.out -e connect bin/ycsb run couchbase2 -P workloads/workloade -p couchbase.host=192.168.10.92 -threads 1 -p maxexecutiontime=120 -p operationcount=1000000
strace -f -o workloada.out -e connect bin/ycsb run couchbase2 -P workloads/workloada -p couchbase.host=192.168.10.112 -threads 4 -p maxexecutiontime=120 -p operationcount=1000000