2019-05-02 11:33:05 INFO GoldenPathTests:56 - #################### Starting set-up Cluster #################### 2019-05-02 11:33:05 INFO GoldenPathTests:75 - cleaning up cluster 10.112.180.103 2019-05-02 11:33:05 INFO RestClient:94 - Cleaning up the cluster: 10.112.180.103 2019-05-02 11:33:05 INFO RestClient:97 - Deleting bucket: default 2019-05-02 11:33:07 INFO RestClient:111 - Rebalancing out nodes: [ns_1@10.112.180.103, ns_1@10.112.180.104] 2019-05-02 11:33:07 INFO GoldenPathTests:78 - Cluster cleanup - DONE 2019-05-02 11:33:10 INFO GoldenPathTests:86 - Adding Node to the cluster 10.112.180.104 2019-05-02 11:33:12 INFO GoldenPathTests:98 - Rebalancing the cluster 2019-05-02 11:33:14 INFO GoldenPathTests:101 - Creating a default bucket 2019-05-02 11:33:19 INFO GoldenPathTests:119 - #################### Starting Test Set_Durability_A[0] #################### 2019-05-02 11:33:19 INFO GoldenPathTests:121 - Connection to the cluster 2019-05-02 11:33:19 DEBUG InternalLoggerFactory:43 - Using SLF4J as the default logging framework 2019-05-02 11:33:19 DEBUG ResourceLeakDetector:129 - -Dcom.couchbase.client.core.deps.io.netty.leakDetection.level: simple 2019-05-02 11:33:19 DEBUG ResourceLeakDetector:130 - -Dcom.couchbase.client.core.deps.io.netty.leakDetection.targetRecords: 4 2019-05-02 11:33:19 DEBUG PlatformDependent:927 - Platform: MacOS 2019-05-02 11:33:19 DEBUG PlatformDependent0:395 - -Dio.netty.noUnsafe: false 2019-05-02 11:33:19 DEBUG PlatformDependent0:851 - Java version: 8 2019-05-02 11:33:19 DEBUG PlatformDependent0:120 - sun.misc.Unsafe.theUnsafe: available 2019-05-02 11:33:19 DEBUG PlatformDependent0:144 - sun.misc.Unsafe.copyMemory: available 2019-05-02 11:33:19 DEBUG PlatformDependent0:182 - java.nio.Buffer.address: available 2019-05-02 11:33:19 DEBUG PlatformDependent0:243 - direct buffer constructor: available 2019-05-02 11:33:19 DEBUG PlatformDependent0:313 - java.nio.Bits.unaligned: available, true 2019-05-02 11:33:19 DEBUG PlatformDependent0:378 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9 2019-05-02 11:33:19 DEBUG PlatformDependent0:385 - java.nio.DirectByteBuffer.(long, int): available 2019-05-02 11:33:19 DEBUG PlatformDependent:953 - sun.misc.Unsafe: available 2019-05-02 11:33:19 DEBUG PlatformDependent:1060 - -Dio.netty.tmpdir: /var/folders/57/4d6j2t4d5mv7t__qzn7zsdrc0000gn/T (java.io.tmpdir) 2019-05-02 11:33:19 DEBUG PlatformDependent:1139 - -Dio.netty.bitMode: 64 (sun.arch.data.model) 2019-05-02 11:33:19 DEBUG PlatformDependent:158 - -Dio.netty.maxDirectMemory: 3817865216 bytes 2019-05-02 11:33:19 DEBUG PlatformDependent:165 - -Dio.netty.uninitializedArrayAllocationThreshold: -1 2019-05-02 11:33:19 DEBUG CleanerJava6:92 - java.nio.ByteBuffer.cleaner(): available 2019-05-02 11:33:19 DEBUG PlatformDependent:185 - -Dio.netty.noPreferDirect: false 2019-05-02 11:33:19 DEBUG ResourceLeakDetectorFactory:202 - Loaded default ResourceLeakDetector: com.couchbase.client.core.deps.io.netty.util.ResourceLeakDetector@66d64fb7 2019-05-02 11:33:19 DEBUG PlatformDependent:821 - com.couchbase.client.core.deps.org.jctools-core.MpscChunkedArrayQueue: available 2019-05-02 11:33:19 DEBUG Loggers$LoggerFactory:249 - Using Slf4j logging framework 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:73 - -Dio.netty.native.workdir: /var/folders/57/4d6j2t4d5mv7t__qzn7zsdrc0000gn/T (io.netty.tmpdir) 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:78 - -Dio.netty.native.deleteLibAfterLoading: true 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:82 - -Dio.netty.native.tryPatchShadedId: true 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:344 - Unable to load the library 'com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64', trying other loading mechanism. java.lang.UnsatisfiedLinkError: no com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) ~[?:1.8.0_131] at java.lang.Runtime.loadLibrary0(Runtime.java:870) ~[?:1.8.0_131] at java.lang.System.loadLibrary(System.java:1122) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) ~[core-io-2.0.0-alpha.2.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369) ~[core-io-2.0.0-alpha.2.jar:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.loadNativeLibrary(Native.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.(Native.java:60) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.KQueue.(KQueue.java:36) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.createEventLoopGroup(IoEnvironment.java:216) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:90) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:43) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment$Builder.build(IoEnvironment.java:280) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.create(IoEnvironment.java:53) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.CoreEnvironment.(CoreEnvironment.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:27) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:24) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment$Builder.build(ClusterEnvironment.java:84) [java-client-3.0.0-alpha.2.jar:?] at com.sanity.java.JavaSDK.connectCluster(JavaSDK.java:59) [classes/:?] at com.sanity.java.GoldenPathTests.setup(GoldenPathTests.java:122) [classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) [junit-4.12.jar:4.12] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:140 - com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64 cannot be loaded from java.libary.path, now trying export to -Dio.netty.native.workdir: /var/folders/57/4d6j2t4d5mv7t__qzn7zsdrc0000gn/T java.lang.UnsatisfiedLinkError: no com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) ~[?:1.8.0_131] at java.lang.Runtime.loadLibrary0(Runtime.java:870) ~[?:1.8.0_131] at java.lang.System.loadLibrary(System.java:1122) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:349) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.loadNativeLibrary(Native.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.(Native.java:60) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.KQueue.(KQueue.java:36) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.createEventLoopGroup(IoEnvironment.java:216) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:90) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:43) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment$Builder.build(IoEnvironment.java:280) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.create(IoEnvironment.java:53) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.CoreEnvironment.(CoreEnvironment.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:27) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:24) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment$Builder.build(ClusterEnvironment.java:84) [java-client-3.0.0-alpha.2.jar:?] at com.sanity.java.JavaSDK.connectCluster(JavaSDK.java:59) [classes/:?] at com.sanity.java.GoldenPathTests.setup(GoldenPathTests.java:122) [classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) [junit-4.12.jar:4.12] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] Suppressed: java.lang.UnsatisfiedLinkError: no com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64 in java.library.path at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867) ~[?:1.8.0_131] at java.lang.Runtime.loadLibrary0(Runtime.java:870) ~[?:1.8.0_131] at java.lang.System.loadLibrary(System.java:1122) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38) ~[core-io-2.0.0-alpha.2.jar:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:369) ~[core-io-2.0.0-alpha.2.jar:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_131] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:361) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:339) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:136) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.loadNativeLibrary(Native.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.Native.(Native.java:60) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.KQueue.(KQueue.java:36) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.createEventLoopGroup(IoEnvironment.java:216) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:90) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.(IoEnvironment.java:43) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment$Builder.build(IoEnvironment.java:280) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.IoEnvironment.create(IoEnvironment.java:53) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.env.CoreEnvironment.(CoreEnvironment.java:125) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:27) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment.(ClusterEnvironment.java:24) [java-client-3.0.0-alpha.2.jar:?] at com.couchbase.client.java.env.ClusterEnvironment$Builder.build(ClusterEnvironment.java:84) [java-client-3.0.0-alpha.2.jar:?] at com.sanity.java.JavaSDK.connectCluster(JavaSDK.java:59) [classes/:?] at com.sanity.java.GoldenPathTests.setup(GoldenPathTests.java:122) [classes/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_131] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_131] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_131] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_131] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) [junit-4.12.jar:4.12] at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) [junit-4.12.jar:4.12] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:319 - Found the ID of the shaded native library com_couchbase_client_core_deps_netty_transport_native_kqueue_x86_64. Replacing ID part netty_transport_native_kqueue_x86_64 with s0cMJK8OF77U8MKIKN88s9EVrPe5UlzajDhm 2019-05-02 11:33:19 DEBUG NativeLibraryLoader:340 - Successfully loaded the library /var/folders/57/4d6j2t4d5mv7t__qzn7zsdrc0000gn/T/libcom_couchbase_client_core_deps_netty_transport_native_kqueue_x86_642173198385477681663.dylib 2019-05-02 11:33:19 DEBUG NetUtil:139 - -Djava.net.preferIPv4Stack: false 2019-05-02 11:33:19 DEBUG NetUtil:140 - -Djava.net.preferIPv6Addresses: false 2019-05-02 11:33:19 DEBUG NetUtil:224 - Loopback interface: lo0 (lo0, 0:0:0:0:0:0:0:1) 2019-05-02 11:33:19 DEBUG NetUtil:289 - Failed to get SOMAXCONN from sysctl and file /proc/sys/net/core/somaxconn. Default: 128 2019-05-02 11:33:19 DEBUG MultithreadEventLoopGroup:44 - -Dio.netty.eventLoopThreads: 16 2019-05-02 11:33:19 INFO JavaSDK:61 - Cluster connection is successful 2019-05-02 11:33:19 DEBUG DefaultChannelId:79 - -Dio.netty.processId: 7090 (auto-detected) 2019-05-02 11:33:19 DEBUG DefaultChannelId:101 - -Dio.netty.machineId: 18:65:90:ff:fe:cd:45:d1 (auto-detected) 2019-05-02 11:33:19 DEBUG InternalThreadLocalMap:54 - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024 2019-05-02 11:33:19 DEBUG InternalThreadLocalMap:57 - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096 2019-05-02 11:33:19 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":1,"remote":"10.112.180.103"} 2019-05-02 11:33:19 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":1,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:128 - -Dio.netty.allocator.numHeapArenas: 16 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:129 - -Dio.netty.allocator.numDirectArenas: 16 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:131 - -Dio.netty.allocator.pageSize: 8192 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:136 - -Dio.netty.allocator.maxOrder: 11 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:140 - -Dio.netty.allocator.chunkSize: 16777216 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:141 - -Dio.netty.allocator.tinyCacheSize: 512 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:142 - -Dio.netty.allocator.smallCacheSize: 256 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:143 - -Dio.netty.allocator.normalCacheSize: 64 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:144 - -Dio.netty.allocator.maxCachedBufferCapacity: 32768 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:145 - -Dio.netty.allocator.cacheTrimInterval: 8192 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:146 - -Dio.netty.allocator.useCacheForAllThreads: true 2019-05-02 11:33:19 DEBUG PooledByteBufAllocator:147 - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023 2019-05-02 11:33:19 DEBUG ByteBufUtil:84 - -Dio.netty.allocator.type: pooled 2019-05-02 11:33:19 DEBUG ByteBufUtil:93 - -Dio.netty.threadLocalDirectBufferSize: 0 2019-05-02 11:33:19 DEBUG ByteBufUtil:96 - -Dio.netty.maxThreadLocalCharBufferSize: 16384 2019-05-02 11:33:19 DEBUG Recycler:96 - -Dio.netty.recycler.maxCapacityPerThread: 4096 2019-05-02 11:33:19 DEBUG Recycler:97 - -Dio.netty.recycler.maxSharedCapacityFactor: 2 2019-05-02 11:33:19 DEBUG Recycler:98 - -Dio.netty.recycler.linkCapacity: 16 2019-05-02 11:33:19 DEBUG Recycler:99 - -Dio.netty.recycler.ratio: 8 2019-05-02 11:33:19 DEBUG AbstractByteBuf:62 - -Dcom.couchbase.client.core.deps.io.netty.buffer.checkAccessible: true 2019-05-02 11:33:19 DEBUG AbstractByteBuf:63 - -Dcom.couchbase.client.core.deps.io.netty.buffer.checkBounds: true 2019-05-02 11:33:19 DEBUG ResourceLeakDetectorFactory:202 - Loaded default ResourceLeakDetector: com.couchbase.client.core.deps.io.netty.util.ResourceLeakDetector@1cc68c8e 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][420991µs] Service added to Node {"bucket":"default","coreId":1,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":1,"requestId":1,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":1,"completed":false} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][47551µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":1,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57979"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][1132µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":1,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57979"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":1,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57979"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":2,"requestId":1,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":1,"completed":false} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][158969µs] SASL Authentication completed {"bucket":"default","coreId":1,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57979"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][704µs] Bucket "default" selected {"bucket":"default","coreId":1,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57979"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][380498µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.001132976S, FeatureNegotiatingHandler=PT0.0475516S, SaslAuthenticationHandler=PT0.158969564S, SelectBucketHandler=PT0.000704887S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":1,"remote":"10.112.180.103:11210","type":"KV","local":"/10.112.180.1:57979","channelId":"0000000000000001/000000006CE40399"} 2019-05-02 11:33:20 INFO GoldenPathTests:153 - Figuring out active and replica nodes 2019-05-02 11:33:20 INFO RemoteHelper:94 - Document:Set_MAJORITY_A, vBucket:747 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [CONFIG][ConfigUpdatedEvent] Bucket configuration updated: DefaultCouchbaseBucketConfig{name='default', locator=VBUCKET, uri='/pools/default/buckets/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', streamingUri='/pools/default/bucketsStreaming/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', nodeInfo=[DefaultNodeInfo{rawHostname='10.112.180.103', hostname=NetworkAddress{10.112.180.103/10.112.180.103, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}, DefaultNodeInfo{rawHostname='10.112.180.104', hostname=NetworkAddress{10.112.180.104/10.112.180.104, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}], partitionInfo=PartitionInfo{numberOfReplicas=1, partitionHosts=[10.112.180.103:11210, 10.112.180.104:11210], partitions=[[0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0]], tainted=false}, tainted=false, rev=1467} {"coreId":1} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][688µs] Service added to Node {"coreId":1,"remote":"10.112.180.103:8091","type":"MANAGER"} 2019-05-02 11:33:20 TRACE CouchbaseLogger:360 - [SERVICE][ServiceAddIgnoredEvent] Service not added to Node. Reason: ALREADY_ADDED {"coreId":1,"remote":"10.112.180.103"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][229µs] Service added to Node {"coreId":1,"remote":"10.112.180.103:8092","type":"VIEWS"} 2019-05-02 11:33:20 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":1,"remote":"10.112.180.104"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][17µs] Service added to Node {"coreId":1,"remote":"10.112.180.104:8091","type":"MANAGER"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":1,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][780µs] Service added to Node {"bucket":"default","coreId":1,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][14µs] Service added to Node {"coreId":1,"remote":"10.112.180.104:8092","type":"VIEWS"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [CORE][ReconfigurationCompletedEvent][13572µs] Reconfiguration attempt completed. {"coreId":1} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][1570µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":1,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57980"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][951µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":1,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57980"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":1,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57980"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][17548µs] SASL Authentication completed {"bucket":"default","coreId":1,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57980"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][605µs] Bucket "default" selected {"bucket":"default","coreId":1,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57980"} 2019-05-02 11:33:20 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][24648µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.000951698S, FeatureNegotiatingHandler=PT0.001570026S, SaslAuthenticationHandler=PT0.017548689S, SelectBucketHandler=PT0.000605332S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":1,"remote":"10.112.180.104:11210","type":"KV","local":"/10.112.180.1:57980","channelId":"0000000000000001/00000000F6343918"} 2019-05-02 11:33:20 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: /opt/couchbase/bin/cbstats 10.112.180.103:11210 -b default -u Administrator -p password vbucket-details 747 2019-05-02 11:33:20 INFO RemoteHelper:76 - vb_747: replica vb_747:backfill_queue_size: 0 vb_747:bloom_filter: ENABLED vb_747:bloom_filter_key_count: 0 vb_747:bloom_filter_size: 95851 vb_747:db_data_size: 243 vb_747:db_file_size: 8238 vb_747:drift_ahead_threshold: 5000000000 vb_747:drift_ahead_threshold_exceeded: 0 vb_747:drift_behind_threshold: 5000000000 vb_747:drift_behind_threshold_exceeded: 0 vb_747:high_prepared_seqno: 0 vb_747:high_seqno: 0 vb_747:hp_vb_req_size: 0 vb_747:ht_cache_size: 0 vb_747:ht_item_memory: 0 vb_747:ht_item_memory_uncompressed: 0 vb_747:ht_memory: 2584 vb_747:ht_size: 47 vb_747:logical_clock_ticks: 0 vb_747:max_cas: 0 vb_747:max_cas_str: 1970-01-01T00:00:00.0 vb_747:max_deleted_revid: 0 vb_747:might_contain_xattrs: false vb_747:num_ejects: 0 vb_747:num_items: 0 vb_747:num_non_resident: 0 vb_747:num_prepared_sync_writes: 0 vb_747:num_temp_items: 0 vb_747:ops_create: 0 vb_747:ops_delete: 0 vb_747:ops_get: 0 vb_747:ops_reject: 0 vb_747:ops_update: 0 vb_747:pending_writes: 0 vb_747:purge_seqno: 0 vb_747:queue_age: 0 vb_747:queue_drain: 2 vb_747:queue_fill: 2 vb_747:queue_memory: 0 vb_747:queue_size: 0 vb_747:rollback_item_count: 0 vb_747:topology: null vb_747:total_abs_drift: 0 vb_747:total_abs_drift_count: 0 vb_747:uuid: 78590778006890 2019-05-02 11:33:20 INFO RemoteHelper:94 - Document:Set_MAJORITY_A, vBucket:747 2019-05-02 11:33:21 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.104 server with command: /opt/couchbase/bin/cbstats 10.112.180.104:11210 -b default -u Administrator -p password vbucket-details 747 2019-05-02 11:33:21 INFO RemoteHelper:76 - vb_747: active vb_747:backfill_queue_size: 0 vb_747:bloom_filter: ENABLED vb_747:bloom_filter_key_count: 0 vb_747:bloom_filter_size: 95851 vb_747:db_data_size: 300 vb_747:db_file_size: 4142 vb_747:drift_ahead_threshold: 5000000000 vb_747:drift_ahead_threshold_exceeded: 0 vb_747:drift_behind_threshold: 5000000000 vb_747:drift_behind_threshold_exceeded: 0 vb_747:high_prepared_seqno: 0 vb_747:high_seqno: 0 vb_747:hp_vb_req_size: 0 vb_747:ht_cache_size: 0 vb_747:ht_item_memory: 0 vb_747:ht_item_memory_uncompressed: 0 vb_747:ht_memory: 2584 vb_747:ht_size: 47 vb_747:logical_clock_ticks: 0 vb_747:max_cas: 0 vb_747:max_cas_str: 1970-01-01T00:00:00.0 vb_747:max_deleted_revid: 0 vb_747:might_contain_xattrs: false vb_747:num_ejects: 0 vb_747:num_items: 0 vb_747:num_non_resident: 0 vb_747:num_prepared_sync_writes: 0 vb_747:num_temp_items: 0 vb_747:ops_create: 0 vb_747:ops_delete: 0 vb_747:ops_get: 0 vb_747:ops_reject: 0 vb_747:ops_update: 0 vb_747:pending_writes: 0 vb_747:purge_seqno: 0 vb_747:queue_age: 0 vb_747:queue_drain: 1 vb_747:queue_fill: 1 vb_747:queue_memory: 0 vb_747:queue_size: 0 vb_747:rollback_item_count: 0 vb_747:topology: [["ns_1@10.112.180.104","ns_1@10.112.180.103"]] vb_747:total_abs_drift: 0 vb_747:total_abs_drift_count: 0 vb_747:uuid: 78590778006890 2019-05-02 11:33:21 INFO GoldenPathTests:164 - Active node: 10.112.180.104:8091 2019-05-02 11:33:21 INFO GoldenPathTests:165 - Replica node: 10.112.180.103:8091 2019-05-02 11:33:21 INFO GoldenPathTests:175 - Insert a document for the very first time 2019-05-02 11:33:21 INFO JavaSDK:101 - Inserting Set_MAJORITY_A 2019-05-02 11:33:21 INFO GoldenPathTests:178 - Start a reader thread to continuously check the document for older value 2019-05-02 11:33:21 INFO GoldenPathTests:184 - Stop memcached on replica node 2019-05-02 11:33:21 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: kill -s STOP `pgrep memcached` 2019-05-02 11:33:21 INFO GoldenPathTests:188 - memcached on replica node is stopped 2019-05-02 11:33:21 INFO JavaSDK:163 - Awaiting for latch 2019-05-02 11:33:21 INFO JavaSDK:208 - Upserting Set_MAJORITY_A 2019-05-02 11:33:21 INFO GoldenPathTests:195 - Asserting the old doc value before memcached starts 2019-05-02 11:33:21 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: kill -s CONT `pgrep memcached` 2019-05-02 11:33:21 INFO JavaSDK:219 - Upsert completed 2019-05-02 11:33:21 INFO GoldenPathTests:205 - Verifying the updated value after memcached resumes 2019-05-02 11:33:21 INFO GoldenPathTests:128 - #################### Finished Test Set_Durability_A[0] #################### 2019-05-02 11:33:21 DEBUG PoolThreadCache:249 - Freed 11 thread-local buffer(s) from thread: cb-io-kv-5-1 2019-05-02 11:33:21 DEBUG PoolThreadCache:249 - Freed 29 thread-local buffer(s) from thread: cb-io-kv-5-2 2019-05-02 11:33:21 INFO GoldenPathTests:119 - #################### Starting Test Set_Durability_A[1] #################### 2019-05-02 11:33:21 INFO GoldenPathTests:121 - Connection to the cluster 2019-05-02 11:33:21 INFO JavaSDK:61 - Cluster connection is successful 2019-05-02 11:33:21 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":2,"remote":"10.112.180.103"} 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":2,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][3216µs] Service added to Node {"bucket":"default","coreId":2,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":1,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][3789µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57985"} 2019-05-02 11:33:21 WARN CouchbaseLogger:420 - [IO][GenericFailureDetectedEvent] Detected Exception in IO Layer: syscall:read(..) failed: Connection reset by peer, Cause: (none) 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":2,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:21 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":3,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:22 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":4,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:22 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":5,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:22 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":6,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:22 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":7,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:22 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":8,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:23 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":9,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:23 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":10,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:23 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":11,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:23 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":12,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:23 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":13,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:24 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":14,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:24 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":15,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:24 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":16,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:24 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":17,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:24 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":18,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:25 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":19,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:25 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":20,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:25 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":21,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:25 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":22,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:25 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":23,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":24,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":25,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:26 DEBUG DefaultRetry:254 - Scheduling retry attempt, retry context: iteration=1 exception=java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 5000ms in 'source(MonoDefer)' (and no fallback has been configured) backoff={32ms/4096ms} 2019-05-02 11:33:26 ERROR Operators:319 - Operator called default onErrorDropped java.util.concurrent.TimeoutException: KV Error Map loading timed out after 5000ms at com.couchbase.client.core.io.netty.kv.ErrorMapLoadingHandler.lambda$channelActive$1(ErrorMapLoadingHandler.java:140) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:263) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [core-io-2.0.0-alpha.2.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":26,"requestId":164,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":2,"completed":false} 2019-05-02 11:33:26 WARN CouchbaseLogger:420 - [ENDPOINT][EndpointConnectionFailedEvent][5000000µs] Connect attempt 1 failed because of TimeoutException: Did not observe any item or terminal signal within 5000ms in 'source(MonoDefer)' (and no fallback has been configured) {"bucket":"default","circuitBreaker":"DISABLED","coreId":2,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][1828µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57986"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][1092µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57986"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57986"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][39455µs] SASL Authentication completed {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57986"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][553µs] Bucket "default" selected {"bucket":"default","coreId":2,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57986"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][45231µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.001092965S, FeatureNegotiatingHandler=PT0.001828114S, SaslAuthenticationHandler=PT0.039455779S, SelectBucketHandler=PT0.000553844S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":2,"remote":"10.112.180.103:11210","type":"KV","local":"/10.112.180.1:57986","channelId":"0000000000000002/00000000CB3A02A0"} 2019-05-02 11:33:26 INFO GoldenPathTests:153 - Figuring out active and replica nodes 2019-05-02 11:33:26 INFO RemoteHelper:94 - Document:Set_MAJORITY_AND_PERSIST_ON_MASTER_A, vBucket:996 2019-05-02 11:33:26 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: /opt/couchbase/bin/cbstats 10.112.180.103:11210 -b default -u Administrator -p password vbucket-details 996 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [CONFIG][ConfigUpdatedEvent] Bucket configuration updated: DefaultCouchbaseBucketConfig{name='default', locator=VBUCKET, uri='/pools/default/buckets/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', streamingUri='/pools/default/bucketsStreaming/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', nodeInfo=[DefaultNodeInfo{rawHostname='10.112.180.103', hostname=NetworkAddress{10.112.180.103/10.112.180.103, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}, DefaultNodeInfo{rawHostname='10.112.180.104', hostname=NetworkAddress{10.112.180.104/10.112.180.104, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}], partitionInfo=PartitionInfo{numberOfReplicas=1, partitionHosts=[10.112.180.103:11210, 10.112.180.104:11210], partitions=[[0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0]], tainted=false}, tainted=false, rev=1467} {"coreId":2} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][22µs] Service added to Node {"coreId":2,"remote":"10.112.180.103:8091","type":"MANAGER"} 2019-05-02 11:33:26 TRACE CouchbaseLogger:360 - [SERVICE][ServiceAddIgnoredEvent] Service not added to Node. Reason: ALREADY_ADDED {"coreId":2,"remote":"10.112.180.103"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][10µs] Service added to Node {"coreId":2,"remote":"10.112.180.103:8092","type":"VIEWS"} 2019-05-02 11:33:26 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":2,"remote":"10.112.180.104"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][9µs] Service added to Node {"coreId":2,"remote":"10.112.180.104:8091","type":"MANAGER"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":2,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][504µs] Service added to Node {"bucket":"default","coreId":2,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][16µs] Service added to Node {"coreId":2,"remote":"10.112.180.104:8092","type":"VIEWS"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [CORE][ReconfigurationCompletedEvent][2011µs] Reconfiguration attempt completed. {"coreId":2} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][1357µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":2,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57987"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][560µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":2,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57987"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":2,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57987"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][14913µs] SASL Authentication completed {"bucket":"default","coreId":2,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57987"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][403µs] Bucket "default" selected {"bucket":"default","coreId":2,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57987"} 2019-05-02 11:33:26 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][19331µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.000560851S, FeatureNegotiatingHandler=PT0.001357722S, SaslAuthenticationHandler=PT0.014913373S, SelectBucketHandler=PT0.000403236S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":2,"remote":"10.112.180.104:11210","type":"KV","local":"/10.112.180.1:57987","channelId":"0000000000000002/00000000F256B201"} 2019-05-02 11:33:26 INFO RemoteHelper:76 - vb_996: replica vb_996:backfill_queue_size: 0 vb_996:bloom_filter: DOESN'T EXIST vb_996:bloom_filter_key_count: 0 vb_996:bloom_filter_size: 0 vb_996:db_data_size: 249 vb_996:db_file_size: 8238 vb_996:drift_ahead_threshold: 5000000000 vb_996:drift_ahead_threshold_exceeded: 0 vb_996:drift_behind_threshold: 5000000000 vb_996:drift_behind_threshold_exceeded: 0 vb_996:high_prepared_seqno: 0 vb_996:high_seqno: 0 vb_996:hp_vb_req_size: 0 vb_996:ht_cache_size: 0 vb_996:ht_item_memory: 0 vb_996:ht_item_memory_uncompressed: 0 vb_996:ht_memory: 2584 vb_996:ht_size: 47 vb_996:logical_clock_ticks: 0 vb_996:max_cas: 0 vb_996:max_cas_str: 1970-01-01T00:00:00.0 vb_996:max_deleted_revid: 0 vb_996:might_contain_xattrs: false vb_996:num_ejects: 0 vb_996:num_items: 0 vb_996:num_non_resident: 0 vb_996:num_prepared_sync_writes: 0 vb_996:num_temp_items: 0 vb_996:ops_create: 0 vb_996:ops_delete: 0 vb_996:ops_get: 0 vb_996:ops_reject: 0 vb_996:ops_update: 0 vb_996:pending_writes: 0 vb_996:purge_seqno: 0 vb_996:queue_age: 0 vb_996:queue_drain: 2 vb_996:queue_fill: 2 vb_996:queue_memory: 0 vb_996:queue_size: 0 vb_996:rollback_item_count: 0 vb_996:topology: null vb_996:total_abs_drift: 0 vb_996:total_abs_drift_count: 0 vb_996:uuid: 174477703824657 2019-05-02 11:33:26 INFO RemoteHelper:94 - Document:Set_MAJORITY_AND_PERSIST_ON_MASTER_A, vBucket:996 2019-05-02 11:33:26 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.104 server with command: /opt/couchbase/bin/cbstats 10.112.180.104:11210 -b default -u Administrator -p password vbucket-details 996 2019-05-02 11:33:27 INFO RemoteHelper:76 - vb_996: active vb_996:backfill_queue_size: 0 vb_996:bloom_filter: ENABLED vb_996:bloom_filter_key_count: 0 vb_996:bloom_filter_size: 95851 vb_996:db_data_size: 300 vb_996:db_file_size: 4142 vb_996:drift_ahead_threshold: 5000000000 vb_996:drift_ahead_threshold_exceeded: 0 vb_996:drift_behind_threshold: 5000000000 vb_996:drift_behind_threshold_exceeded: 0 vb_996:high_prepared_seqno: 0 vb_996:high_seqno: 0 vb_996:hp_vb_req_size: 0 vb_996:ht_cache_size: 0 vb_996:ht_item_memory: 0 vb_996:ht_item_memory_uncompressed: 0 vb_996:ht_memory: 2584 vb_996:ht_size: 47 vb_996:logical_clock_ticks: 0 vb_996:max_cas: 0 vb_996:max_cas_str: 1970-01-01T00:00:00.0 vb_996:max_deleted_revid: 0 vb_996:might_contain_xattrs: false vb_996:num_ejects: 0 vb_996:num_items: 0 vb_996:num_non_resident: 0 vb_996:num_prepared_sync_writes: 0 vb_996:num_temp_items: 0 vb_996:ops_create: 0 vb_996:ops_delete: 0 vb_996:ops_get: 0 vb_996:ops_reject: 0 vb_996:ops_update: 0 vb_996:pending_writes: 0 vb_996:purge_seqno: 0 vb_996:queue_age: 0 vb_996:queue_drain: 1 vb_996:queue_fill: 1 vb_996:queue_memory: 0 vb_996:queue_size: 0 vb_996:rollback_item_count: 0 vb_996:topology: [["ns_1@10.112.180.104","ns_1@10.112.180.103"]] vb_996:total_abs_drift: 0 vb_996:total_abs_drift_count: 0 vb_996:uuid: 174477703824657 2019-05-02 11:33:27 INFO GoldenPathTests:164 - Active node: 10.112.180.104:8091 2019-05-02 11:33:27 INFO GoldenPathTests:165 - Replica node: 10.112.180.103:8091 2019-05-02 11:33:27 INFO GoldenPathTests:175 - Insert a document for the very first time 2019-05-02 11:33:27 INFO JavaSDK:101 - Inserting Set_MAJORITY_AND_PERSIST_ON_MASTER_A 2019-05-02 11:33:27 INFO GoldenPathTests:178 - Start a reader thread to continuously check the document for older value 2019-05-02 11:33:27 INFO GoldenPathTests:184 - Stop memcached on replica node 2019-05-02 11:33:27 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: kill -s STOP `pgrep memcached` 2019-05-02 11:33:27 INFO GoldenPathTests:188 - memcached on replica node is stopped 2019-05-02 11:33:27 INFO JavaSDK:208 - Upserting Set_MAJORITY_AND_PERSIST_ON_MASTER_A 2019-05-02 11:33:27 INFO JavaSDK:163 - Awaiting for latch 2019-05-02 11:33:27 INFO GoldenPathTests:195 - Asserting the old doc value before memcached starts 2019-05-02 11:33:27 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: kill -s CONT `pgrep memcached` 2019-05-02 11:33:27 INFO JavaSDK:219 - Upsert completed 2019-05-02 11:33:27 INFO GoldenPathTests:205 - Verifying the updated value after memcached resumes 2019-05-02 11:33:27 INFO GoldenPathTests:128 - #################### Finished Test Set_Durability_A[1] #################### 2019-05-02 11:33:27 DEBUG PoolThreadCache:249 - Freed 5 thread-local buffer(s) from thread: cb-io-kv-9-1 2019-05-02 11:33:27 DEBUG PoolThreadCache:249 - Freed 28 thread-local buffer(s) from thread: cb-io-kv-9-3 2019-05-02 11:33:27 DEBUG PoolThreadCache:249 - Freed 11 thread-local buffer(s) from thread: cb-io-kv-9-2 2019-05-02 11:33:27 INFO GoldenPathTests:119 - #################### Starting Test Set_Durability_A[2] #################### 2019-05-02 11:33:27 INFO GoldenPathTests:121 - Connection to the cluster 2019-05-02 11:33:27 INFO JavaSDK:61 - Cluster connection is successful 2019-05-02 11:33:27 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":3,"remote":"10.112.180.103"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":3,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][1150µs] Service added to Node {"bucket":"default","coreId":3,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":1,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][13493µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57992"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][2854µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57992"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57992"} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":2,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":3,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:27 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":4,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:28 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":5,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:28 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":6,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:28 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":7,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:28 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":8,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:28 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":9,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:29 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":10,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:29 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":11,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:29 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":12,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:29 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":13,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:29 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":14,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:30 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":15,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:30 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":16,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:30 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":17,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:30 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":18,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:30 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":19,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:31 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":20,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:31 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":21,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:31 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":22,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:31 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":23,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:31 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":24,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":25,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:32 DEBUG DefaultRetry:254 - Scheduling retry attempt, retry context: iteration=1 exception=java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 5000ms in 'source(MonoDefer)' (and no fallback has been configured) backoff={32ms/4096ms} 2019-05-02 11:33:32 ERROR Operators:319 - Operator called default onErrorDropped java.util.concurrent.TimeoutException: KV SASL Negotiation timed out after 5000ms at com.couchbase.client.core.io.netty.kv.SaslAuthenticationHandler.lambda$channelActive$1(SaslAuthenticationHandler.java:163) ~[core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:127) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:263) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) [core-io-2.0.0-alpha.2.jar:?] at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [core-io-2.0.0-alpha.2.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [REQUEST][RequestRetriedEvent][100000µs] Request CarrierBucketConfigRequest retried per RetryStrategy {"retried":26,"requestId":287,"timeoutMs":10000,"service":{"bucket":"default","type":"kv","key":""},"coreId":3,"completed":false} 2019-05-02 11:33:32 WARN CouchbaseLogger:420 - [ENDPOINT][EndpointConnectionFailedEvent][5000000µs] Connect attempt 1 failed because of TimeoutException: Did not observe any item or terminal signal within 5000ms in 'source(MonoDefer)' (and no fallback has been configured) {"bucket":"default","circuitBreaker":"DISABLED","coreId":3,"remote":"10.112.180.103:11210","type":"KV"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][9062µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57993"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][2588µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57993"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57993"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][14859µs] SASL Authentication completed {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57993"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][386µs] Bucket "default" selected {"bucket":"default","coreId":3,"remote":"10.112.180.103/10.112.180.103:11210","local":"/10.112.180.1:57993"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][29003µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.002588616S, FeatureNegotiatingHandler=PT0.009062798S, SaslAuthenticationHandler=PT0.014859623S, SelectBucketHandler=PT0.000386133S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":3,"remote":"10.112.180.103:11210","type":"KV","local":"/10.112.180.1:57993","channelId":"0000000000000003/000000008719216E"} 2019-05-02 11:33:32 INFO GoldenPathTests:153 - Figuring out active and replica nodes 2019-05-02 11:33:32 INFO RemoteHelper:94 - Document:Set_PERSIST_TO_MAJORITY_A, vBucket:71 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [CONFIG][ConfigUpdatedEvent] Bucket configuration updated: DefaultCouchbaseBucketConfig{name='default', locator=VBUCKET, uri='/pools/default/buckets/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', streamingUri='/pools/default/bucketsStreaming/default?bucket_uuid=8addafdca6001f2cb5669dd7c3428026', nodeInfo=[DefaultNodeInfo{rawHostname='10.112.180.103', hostname=NetworkAddress{10.112.180.103/10.112.180.103, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}, DefaultNodeInfo{rawHostname='10.112.180.104', hostname=NetworkAddress{10.112.180.104/10.112.180.104, fromHostname=false, reverseDns=true}, directServices={MANAGER=8091, KV=11210, VIEWS=8092}, sslServices={MANAGER=18091, KV=11207, VIEWS=18092}, alternateAddresses={}, configPort=0}], partitionInfo=PartitionInfo{numberOfReplicas=1, partitionHosts=[10.112.180.103:11210, 10.112.180.104:11210], partitions=[[0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [0,1], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0], [1,0]], tainted=false}, tainted=false, rev=1467} {"coreId":3} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][36µs] Service added to Node {"coreId":3,"remote":"10.112.180.103:8091","type":"MANAGER"} 2019-05-02 11:33:32 TRACE CouchbaseLogger:360 - [SERVICE][ServiceAddIgnoredEvent] Service not added to Node. Reason: ALREADY_ADDED {"coreId":3,"remote":"10.112.180.103"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][18µs] Service added to Node {"coreId":3,"remote":"10.112.180.103:8092","type":"VIEWS"} 2019-05-02 11:33:32 INFO CouchbaseLogger:400 - [NODE][NodeConnectedEvent] Node connected {"coreId":3,"remote":"10.112.180.104"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][12µs] Service added to Node {"coreId":3,"remote":"10.112.180.104:8091","type":"MANAGER"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceConnectInitiatedEvent] Starting to connect service with 1 minimum endpoints {"bucket":"default","coreId":3,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][620µs] Service added to Node {"bucket":"default","coreId":3,"remote":"10.112.180.104:11210","type":"KV"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [SERVICE][ServiceAddedEvent][11µs] Service added to Node {"coreId":3,"remote":"10.112.180.104:8092","type":"VIEWS"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [CORE][ReconfigurationCompletedEvent][2173µs] Reconfiguration attempt completed. {"coreId":3} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][FeaturesNegotiatedEvent][1514µs] Negotiated [XATTR, XERROR, SELECT_BUCKET, SNAPPY, ALT_REQUEST, SYNC_REPLICATION] {"bucket":"default","coreId":3,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57994"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][ErrorMapLoadedEvent][623µs] KV Error Map successfully loaded. Map Version: 1, Revision: 2 {"bucket":"default","coreId":3,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57994"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SaslMechanismsSelectedEvent] SASL Mechanism SCRAM_SHA512 selected. Server: [SCRAM_SHA256, SCRAM_SHA512, SCRAM_SHA1, PLAIN], Allowed: [PLAIN, CRAM_MD5, SCRAM_SHA1, SCRAM_SHA256, SCRAM_SHA512] {"bucket":"default","coreId":3,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57994"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SaslAuthenticationCompletedEvent][16318µs] SASL Authentication completed {"bucket":"default","coreId":3,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57994"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [IO][SelectBucketCompletedEvent][453µs] Bucket "default" selected {"bucket":"default","coreId":3,"remote":"10.112.180.104/10.112.180.104:11210","local":"/10.112.180.1:57994"} 2019-05-02 11:33:32 DEBUG CouchbaseLogger:380 - [ENDPOINT][EndpointConnectedEvent][21193µs] Endpoint connected successfully (Timings: {ErrorMapLoadingHandler=PT0.000623068S, FeatureNegotiatingHandler=PT0.001514452S, SaslAuthenticationHandler=PT0.016318899S, SelectBucketHandler=PT0.000453636S}) {"bucket":"default","circuitBreaker":"DISABLED","coreId":3,"remote":"10.112.180.104:11210","type":"KV","local":"/10.112.180.1:57994","channelId":"0000000000000003/00000000C1F801DA"} 2019-05-02 11:33:32 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.103 server with command: /opt/couchbase/bin/cbstats 10.112.180.103:11210 -b default -u Administrator -p password vbucket-details 71 2019-05-02 11:33:32 INFO RemoteHelper:76 - vb_71: active vb_71:backfill_queue_size: 0 vb_71:bloom_filter: DOESN'T EXIST vb_71:bloom_filter_key_count: 0 vb_71:bloom_filter_size: 0 vb_71:db_data_size: 338 vb_71:db_file_size: 12334 vb_71:drift_ahead_threshold: 5000000000 vb_71:drift_ahead_threshold_exceeded: 0 vb_71:drift_behind_threshold: 5000000000 vb_71:drift_behind_threshold_exceeded: 0 vb_71:high_prepared_seqno: 0 vb_71:high_seqno: 0 vb_71:hp_vb_req_size: 0 vb_71:ht_cache_size: 0 vb_71:ht_item_memory: 0 vb_71:ht_item_memory_uncompressed: 0 vb_71:ht_memory: 2584 vb_71:ht_size: 47 vb_71:logical_clock_ticks: 0 vb_71:max_cas: 0 vb_71:max_cas_str: 1970-01-01T00:00:00.0 vb_71:max_deleted_revid: 0 vb_71:might_contain_xattrs: false vb_71:num_ejects: 0 vb_71:num_items: 0 vb_71:num_non_resident: 0 vb_71:num_prepared_sync_writes: 0 vb_71:num_temp_items: 0 vb_71:ops_create: 0 vb_71:ops_delete: 0 vb_71:ops_get: 0 vb_71:ops_reject: 0 vb_71:ops_update: 0 vb_71:pending_writes: 0 vb_71:purge_seqno: 0 vb_71:queue_age: 0 vb_71:queue_drain: 1 vb_71:queue_fill: 1 vb_71:queue_memory: 0 vb_71:queue_size: 0 vb_71:rollback_item_count: 0 vb_71:topology: [["ns_1@10.112.180.103","ns_1@10.112.180.104"]] vb_71:total_abs_drift: 0 vb_71:total_abs_drift_count: 0 vb_71:uuid: 105620715028013 2019-05-02 11:33:32 INFO RemoteHelper:94 - Document:Set_PERSIST_TO_MAJORITY_A, vBucket:71 2019-05-02 11:33:32 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.104 server with command: /opt/couchbase/bin/cbstats 10.112.180.104:11210 -b default -u Administrator -p password vbucket-details 71 2019-05-02 11:33:32 INFO RemoteHelper:76 - vb_71: replica vb_71:backfill_queue_size: 0 vb_71:bloom_filter: ENABLED vb_71:bloom_filter_key_count: 0 vb_71:bloom_filter_size: 95851 vb_71:db_data_size: 287 vb_71:db_file_size: 16430 vb_71:drift_ahead_threshold: 5000000000 vb_71:drift_ahead_threshold_exceeded: 0 vb_71:drift_behind_threshold: 5000000000 vb_71:drift_behind_threshold_exceeded: 0 vb_71:high_prepared_seqno: 0 vb_71:high_seqno: 0 vb_71:hp_vb_req_size: 0 vb_71:ht_cache_size: 0 vb_71:ht_item_memory: 0 vb_71:ht_item_memory_uncompressed: 0 vb_71:ht_memory: 2584 vb_71:ht_size: 47 vb_71:logical_clock_ticks: 0 vb_71:max_cas: 0 vb_71:max_cas_str: 1970-01-01T00:00:00.0 vb_71:max_deleted_revid: 0 vb_71:might_contain_xattrs: false vb_71:num_ejects: 0 vb_71:num_items: 0 vb_71:num_non_resident: 0 vb_71:num_prepared_sync_writes: 0 vb_71:num_temp_items: 0 vb_71:ops_create: 0 vb_71:ops_delete: 0 vb_71:ops_get: 0 vb_71:ops_reject: 0 vb_71:ops_update: 0 vb_71:pending_writes: 0 vb_71:purge_seqno: 0 vb_71:queue_age: 0 vb_71:queue_drain: 4 vb_71:queue_fill: 4 vb_71:queue_memory: 0 vb_71:queue_size: 0 vb_71:rollback_item_count: 0 vb_71:topology: null vb_71:total_abs_drift: 0 vb_71:total_abs_drift_count: 0 vb_71:uuid: 105620715028013 2019-05-02 11:33:32 INFO GoldenPathTests:164 - Active node: 10.112.180.103:8091 2019-05-02 11:33:32 INFO GoldenPathTests:165 - Replica node: 10.112.180.104:8091 2019-05-02 11:33:32 INFO GoldenPathTests:175 - Insert a document for the very first time 2019-05-02 11:33:32 INFO JavaSDK:101 - Inserting Set_PERSIST_TO_MAJORITY_A 2019-05-02 11:33:32 INFO GoldenPathTests:178 - Start a reader thread to continuously check the document for older value 2019-05-02 11:33:32 INFO GoldenPathTests:184 - Stop memcached on replica node 2019-05-02 11:33:32 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.104 server with command: kill -s STOP `pgrep memcached` 2019-05-02 11:33:32 INFO GoldenPathTests:188 - memcached on replica node is stopped 2019-05-02 11:33:32 INFO JavaSDK:208 - Upserting Set_PERSIST_TO_MAJORITY_A 2019-05-02 11:33:32 INFO JavaSDK:163 - Awaiting for latch 2019-05-02 11:33:32 INFO GoldenPathTests:195 - Asserting the old doc value before memcached starts 2019-05-02 11:33:32 INFO RemoteHelper:69 - Channel Connected to machine 10.112.180.104 server with command: kill -s CONT `pgrep memcached` 2019-05-02 11:33:32 INFO JavaSDK:219 - Upsert completed 2019-05-02 11:33:32 INFO GoldenPathTests:205 - Verifying the updated value after memcached resumes 2019-05-02 11:33:32 INFO GoldenPathTests:128 - #################### Finished Test Set_Durability_A[2] #################### 2019-05-02 11:33:32 DEBUG PoolThreadCache:249 - Freed 11 thread-local buffer(s) from thread: cb-io-kv-13-1 2019-05-02 11:33:32 DEBUG PoolThreadCache:249 - Freed 29 thread-local buffer(s) from thread: cb-io-kv-13-2 2019-05-02 11:33:32 DEBUG PoolThreadCache:249 - Freed 10 thread-local buffer(s) from thread: cb-io-kv-13-3