Uploaded image for project: 'Couchbase Java Client'
  1. Couchbase Java Client
  2. JCBC-646

High latency when using ReplicateTo with 2.0 Java SDK

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 2.1.0
    • 2.0.1
    • Core
    • Security Level: Public
    • None
    • Couchbase Server 3.0.1 / Ubuntu 12.04 / Java SDK 2.0.1

    Description

      I have been trying out some test cases which make use of ReplicateTo in preparation for a customer evaluation.

      I found the latency for write operations using ReplicateTo=1 with Java SDK 2.0 quite high.
      In my test environment I am getting a 95th percentile of 35ms for operation completion.

      For comparison, I also ran the replication test tool created for the C SDK, this has a 95th percentile latency of around 3-4ms in the same environment.
      With the Java SDK 1.4.6, the 95th percentile latency is again around 35ms (with the default obsPollInt).

      When the same Java 2.0 SDK test is run, but without ReplicateTo, then 95th percentile latency is 1ms.

      There are some slight differences between each of the test cases in order for them to work with the different SDKs, but I think they should all capture the same intent of measuring ReplicateTo latency.

      My test cases are available here:
      https://github.com/tom-cb/cb-java2

      SDK14 and SDK20 directories have the test cases for each SDK.
      To build and run the each test case:
      $ cd SDKXX
      $ ./compile-cmd
      $ ./run-cmd

      NB: you will obviously need to edit the main function to have the correct IP address for your CB cluster.

      What the test case does:

      Loop for 1,000 iterations:
      issue an asynchronous set with ReplicateTo.ONE
      sleep for 100ms

      if any operations TimeOut, issue a retry TimeOut of 500ms is used.
      TimedOut operations are discarded from the timing results.

      block until all 1,000 operations are complete

      Print out latency for each operation, and the latency at the 95th Percentile.

      Tail of sample output:
      ... ... ...
      Write time for key: key-696-batch-0 in ms: 36
      Write time for key: key-501-batch-0 in ms: 31
      Write time for key: key-861-batch-0 in ms: 31
      Write time for key: key-437-batch-0 in ms: 31
      Write time for key: key-567-batch-0 in ms: 31
      95th Percentile: 35.0

      Details of the C SDK tool can be found in MB-11173.
      Command used to run the C test:
      $ gcc -O2 -o durability-test-simple durability-test-simple.c -l couchbase
      $ ./durability-test-simple 1 1 1000 2 0

      For the Java test cases, I also tried with a much higher iteration count (100,000) in case it was just an issue with the short running loop not giving HotSpot a chance to warm up. Results overall did look improved, but the 95th Percentile was still around 35ms.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              daschl Michael Nitschinger
              tom.green Tom Green (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty